[PULL 00/29] migration queue

Dr. David Alan Gilbert (git) posted 29 patches 1 year, 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220719170221.576190-1-dgilbert@redhat.com
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, Eduardo Habkost <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Yanan Wang <wangyanan55@huawei.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Juan Quintela <quintela@redhat.com>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>, Thomas Huth <thuth@redhat.com>, Laurent Vivier <lvivier@redhat.com>
There is a newer version of this series
accel/kvm/kvm-all.c             |  46 ++-
accel/stubs/kvm-stub.c          |   5 +
cpus-common.c                   |   8 +
hmp-commands-info.hx            |  13 +
hmp-commands.hx                 |  32 +++
include/exec/cpu-common.h       |   1 +
include/exec/memory.h           |   5 +-
include/hw/core/cpu.h           |   6 +
include/monitor/hmp.h           |   3 +
include/sysemu/dirtylimit.h     |  37 +++
include/sysemu/dirtyrate.h      |  28 ++
include/sysemu/kvm.h            |   2 +
io/channel-socket.c             |   8 +-
migration/channel.c             |   9 +-
migration/dirtyrate.c           | 227 +++++++++------
migration/dirtyrate.h           |   7 +-
migration/migration.c           | 152 ++++++++--
migration/migration.h           |  44 ++-
migration/multifd-zlib.c        |  38 ++-
migration/multifd.c             |   6 +-
migration/multifd.h             |  66 +++--
migration/postcopy-ram.c        | 186 ++++++++++++-
migration/postcopy-ram.h        |  11 +
migration/qemu-file.c           |  31 ++-
migration/qemu-file.h           |   1 +
migration/ram.c                 | 331 ++++++++++++++++++++--
migration/ram.h                 |   6 +-
migration/savevm.c              |  46 ++-
migration/socket.c              |  22 +-
migration/socket.h              |   1 +
migration/tls.c                 |   9 +
migration/tls.h                 |   4 +
migration/trace-events          |  15 +-
monitor/hmp-cmds.c              |   5 +
qapi/migration.json             |  94 ++++++-
softmmu/dirtylimit.c            | 601 ++++++++++++++++++++++++++++++++++++++++
softmmu/meson.build             |   1 +
softmmu/trace-events            |   7 +
tests/qtest/migration-helpers.c |  22 ++
tests/qtest/migration-helpers.h |   2 +
tests/qtest/migration-test.c    | 539 +++++++++++++++++++++++++++++------
tests/qtest/qmp-cmd-test.c      |   2 +
42 files changed, 2394 insertions(+), 285 deletions(-)
create mode 100644 include/sysemu/dirtylimit.h
create mode 100644 include/sysemu/dirtyrate.h
create mode 100644 softmmu/dirtylimit.c
[PULL 00/29] migration queue
Posted by Dr. David Alan Gilbert (git) 1 year, 8 months ago
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:

  Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)

are available in the Git repository at:

  https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c

for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:

  migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)

----------------------------------------------------------------
Migration pull 2022-07-19

  Hyman's dirty page rate limit set
  Ilya's fix for zlib vs migration
  Peter's postcopy-preempt
  Cleanup from Dan
  zero-copy tidy ups from Leo
  multifd doc fix from Juan

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

----------------------------------------------------------------
Daniel P. Berrangé (1):
      migration: remove unreachable code after reading data

Hyman Huang (8):
      accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping
      cpus: Introduce cpu_list_generation_id
      migration/dirtyrate: Refactor dirty page rate calculation
      softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically
      accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function
      softmmu/dirtylimit: Implement virtual CPU throttle
      softmmu/dirtylimit: Implement dirty page rate limit
      tests: Add dirty page rate limit test

Ilya Leoshkevich (1):
      multifd: Copy pages before compressing them with zlib

Juan Quintela (1):
      multifd: Document the locking of MultiFD{Send/Recv}Params

Leonardo Bras (4):
      QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent
      Add dirty-sync-missed-zero-copy migration stat
      migration/multifd: Report to user when zerocopy not working
      migration: Avoid false-positive on non-supported scenarios for zero-copy-send

Peter Xu (14):
      migration: Add postcopy-preempt capability
      migration: Postcopy preemption preparation on channel creation
      migration: Postcopy preemption enablement
      migration: Postcopy recover with preempt enabled
      migration: Create the postcopy preempt channel asynchronously
      migration: Add property x-postcopy-preempt-break-huge
      migration: Add helpers to detect TLS capability
      migration: Export tls-[creds|hostname|authz] params to cmdline too
      migration: Enable TLS for preempt channel
      migration: Respect postcopy request order in preemption mode
      tests: Move MigrateCommon upper
      tests: Add postcopy tls migration test
      tests: Add postcopy tls recovery migration test
      tests: Add postcopy preempt tests

 accel/kvm/kvm-all.c             |  46 ++-
 accel/stubs/kvm-stub.c          |   5 +
 cpus-common.c                   |   8 +
 hmp-commands-info.hx            |  13 +
 hmp-commands.hx                 |  32 +++
 include/exec/cpu-common.h       |   1 +
 include/exec/memory.h           |   5 +-
 include/hw/core/cpu.h           |   6 +
 include/monitor/hmp.h           |   3 +
 include/sysemu/dirtylimit.h     |  37 +++
 include/sysemu/dirtyrate.h      |  28 ++
 include/sysemu/kvm.h            |   2 +
 io/channel-socket.c             |   8 +-
 migration/channel.c             |   9 +-
 migration/dirtyrate.c           | 227 +++++++++------
 migration/dirtyrate.h           |   7 +-
 migration/migration.c           | 152 ++++++++--
 migration/migration.h           |  44 ++-
 migration/multifd-zlib.c        |  38 ++-
 migration/multifd.c             |   6 +-
 migration/multifd.h             |  66 +++--
 migration/postcopy-ram.c        | 186 ++++++++++++-
 migration/postcopy-ram.h        |  11 +
 migration/qemu-file.c           |  31 ++-
 migration/qemu-file.h           |   1 +
 migration/ram.c                 | 331 ++++++++++++++++++++--
 migration/ram.h                 |   6 +-
 migration/savevm.c              |  46 ++-
 migration/socket.c              |  22 +-
 migration/socket.h              |   1 +
 migration/tls.c                 |   9 +
 migration/tls.h                 |   4 +
 migration/trace-events          |  15 +-
 monitor/hmp-cmds.c              |   5 +
 qapi/migration.json             |  94 ++++++-
 softmmu/dirtylimit.c            | 601 ++++++++++++++++++++++++++++++++++++++++
 softmmu/meson.build             |   1 +
 softmmu/trace-events            |   7 +
 tests/qtest/migration-helpers.c |  22 ++
 tests/qtest/migration-helpers.h |   2 +
 tests/qtest/migration-test.c    | 539 +++++++++++++++++++++++++++++------
 tests/qtest/qmp-cmd-test.c      |   2 +
 42 files changed, 2394 insertions(+), 285 deletions(-)
 create mode 100644 include/sysemu/dirtylimit.h
 create mode 100644 include/sysemu/dirtyrate.h
 create mode 100644 softmmu/dirtylimit.c


Re: [PULL 00/29] migration queue
Posted by Peter Maydell 1 year, 8 months ago
On Tue, 19 Jul 2022 at 18:16, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
>
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:
>
>   Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c
>
> for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:
>
>   migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)
>
> ----------------------------------------------------------------
> Migration pull 2022-07-19
>
>   Hyman's dirty page rate limit set
>   Ilya's fix for zlib vs migration
>   Peter's postcopy-preempt
>   Cleanup from Dan
>   zero-copy tidy ups from Leo
>   multifd doc fix from Juan
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>
> ----------------------------------------------------------------

Fails to build on some configs, eg:
https://gitlab.com/qemu-project/qemu/-/jobs/2743059797
https://gitlab.com/qemu-project/qemu/-/jobs/2743059743

../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_tls_psk':
../tests/qtest/migration-test.c:1168:23: error:
'test_migrate_tls_psk_start_match' undeclared (first use in this
function)
1168 | .start_hook = test_migrate_tls_psk_start_match,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c:1168:23: note: each undeclared
identifier is reported only once for each function it appears in
../tests/qtest/migration-test.c:1169:24: error:
'test_migrate_tls_psk_finish' undeclared (first use in this function)
1169 | .finish_hook = test_migrate_tls_psk_finish,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c: In function 'test_postcopy_recovery_tls_psk':
../tests/qtest/migration-test.c:1247:23: error:
'test_migrate_tls_psk_start_match' undeclared (first use in this
function)
1247 | .start_hook = test_migrate_tls_psk_start_match,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c:1248:24: error:
'test_migrate_tls_psk_finish' undeclared (first use in this function)
1248 | .finish_hook = test_migrate_tls_psk_finish,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_all':
../tests/qtest/migration-test.c:1268:23: error:
'test_migrate_tls_psk_start_match' undeclared (first use in this
function)
1268 | .start_hook = test_migrate_tls_psk_start_match,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c:1269:24: error:
'test_migrate_tls_psk_finish' undeclared (first use in this function)
1269 | .finish_hook = test_migrate_tls_psk_finish,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
At top level:
../tests/qtest/migration-test.c:1264:13: error:
'test_postcopy_preempt_all' defined but not used
[-Werror=unused-function]
1264 | static void test_postcopy_preempt_all(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c:1244:13: error:
'test_postcopy_recovery_tls_psk' defined but not used
[-Werror=unused-function]
1244 | static void test_postcopy_recovery_tls_psk(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../tests/qtest/migration-test.c:1164:13: error:
'test_postcopy_preempt_tls_psk' defined but not used
[-Werror=unused-function]
1164 | static void test_postcopy_preempt_tls_psk(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~


-- PMM
Re: [PULL 00/29] migration queue
Posted by Dr. David Alan Gilbert 1 year, 8 months ago
* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Tue, 19 Jul 2022 at 18:16, Dr. David Alan Gilbert (git)
> <dgilbert@redhat.com> wrote:
> >
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:
> >
> >   Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c
> >
> > for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:
> >
> >   migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)
> >
> > ----------------------------------------------------------------
> > Migration pull 2022-07-19
> >
> >   Hyman's dirty page rate limit set
> >   Ilya's fix for zlib vs migration
> >   Peter's postcopy-preempt
> >   Cleanup from Dan
> >   zero-copy tidy ups from Leo
> >   multifd doc fix from Juan
> >
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> >
> > ----------------------------------------------------------------
> 
> Fails to build on some configs, eg:
> https://gitlab.com/qemu-project/qemu/-/jobs/2743059797
> https://gitlab.com/qemu-project/qemu/-/jobs/2743059743

Sorry about that; I thought I'd fixed that but I only got one of the
cases; I'll send a new version shortly.

Dave

> ../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_tls_psk':
> ../tests/qtest/migration-test.c:1168:23: error:
> 'test_migrate_tls_psk_start_match' undeclared (first use in this
> function)
> 1168 | .start_hook = test_migrate_tls_psk_start_match,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1168:23: note: each undeclared
> identifier is reported only once for each function it appears in
> ../tests/qtest/migration-test.c:1169:24: error:
> 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> 1169 | .finish_hook = test_migrate_tls_psk_finish,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c: In function 'test_postcopy_recovery_tls_psk':
> ../tests/qtest/migration-test.c:1247:23: error:
> 'test_migrate_tls_psk_start_match' undeclared (first use in this
> function)
> 1247 | .start_hook = test_migrate_tls_psk_start_match,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1248:24: error:
> 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> 1248 | .finish_hook = test_migrate_tls_psk_finish,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_all':
> ../tests/qtest/migration-test.c:1268:23: error:
> 'test_migrate_tls_psk_start_match' undeclared (first use in this
> function)
> 1268 | .start_hook = test_migrate_tls_psk_start_match,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1269:24: error:
> 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> 1269 | .finish_hook = test_migrate_tls_psk_finish,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> At top level:
> ../tests/qtest/migration-test.c:1264:13: error:
> 'test_postcopy_preempt_all' defined but not used
> [-Werror=unused-function]
> 1264 | static void test_postcopy_preempt_all(void)
> | ^~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1244:13: error:
> 'test_postcopy_recovery_tls_psk' defined but not used
> [-Werror=unused-function]
> 1244 | static void test_postcopy_recovery_tls_psk(void)
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1164:13: error:
> 'test_postcopy_preempt_tls_psk' defined but not used
> [-Werror=unused-function]
> 1164 | static void test_postcopy_preempt_tls_psk(void)
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> 
> -- PMM
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: [PULL 00/29] migration queue
Posted by Peter Xu 1 year, 8 months ago
On Tue, Jul 19, 2022 at 10:53:33PM +0100, Peter Maydell wrote:
> On Tue, 19 Jul 2022 at 18:16, Dr. David Alan Gilbert (git)
> <dgilbert@redhat.com> wrote:
> >
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:
> >
> >   Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c
> >
> > for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:
> >
> >   migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)
> >
> > ----------------------------------------------------------------
> > Migration pull 2022-07-19
> >
> >   Hyman's dirty page rate limit set
> >   Ilya's fix for zlib vs migration
> >   Peter's postcopy-preempt
> >   Cleanup from Dan
> >   zero-copy tidy ups from Leo
> >   multifd doc fix from Juan
> >
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> >
> > ----------------------------------------------------------------
> 
> Fails to build on some configs, eg:
> https://gitlab.com/qemu-project/qemu/-/jobs/2743059797
> https://gitlab.com/qemu-project/qemu/-/jobs/2743059743
> 
> ../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_tls_psk':
> ../tests/qtest/migration-test.c:1168:23: error:
> 'test_migrate_tls_psk_start_match' undeclared (first use in this
> function)
> 1168 | .start_hook = test_migrate_tls_psk_start_match,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1168:23: note: each undeclared
> identifier is reported only once for each function it appears in
> ../tests/qtest/migration-test.c:1169:24: error:
> 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> 1169 | .finish_hook = test_migrate_tls_psk_finish,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c: In function 'test_postcopy_recovery_tls_psk':
> ../tests/qtest/migration-test.c:1247:23: error:
> 'test_migrate_tls_psk_start_match' undeclared (first use in this
> function)
> 1247 | .start_hook = test_migrate_tls_psk_start_match,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1248:24: error:
> 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> 1248 | .finish_hook = test_migrate_tls_psk_finish,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_all':
> ../tests/qtest/migration-test.c:1268:23: error:
> 'test_migrate_tls_psk_start_match' undeclared (first use in this
> function)
> 1268 | .start_hook = test_migrate_tls_psk_start_match,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1269:24: error:
> 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> 1269 | .finish_hook = test_migrate_tls_psk_finish,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> At top level:
> ../tests/qtest/migration-test.c:1264:13: error:
> 'test_postcopy_preempt_all' defined but not used
> [-Werror=unused-function]
> 1264 | static void test_postcopy_preempt_all(void)
> | ^~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1244:13: error:
> 'test_postcopy_recovery_tls_psk' defined but not used
> [-Werror=unused-function]
> 1244 | static void test_postcopy_recovery_tls_psk(void)
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../tests/qtest/migration-test.c:1164:13: error:
> 'test_postcopy_preempt_tls_psk' defined but not used
> [-Werror=unused-function]
> 1164 | static void test_postcopy_preempt_tls_psk(void)
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Sorry my fault.  We'll need to fix the 3 test patches one by one to use "#ifdef
CONFIG_GNUTLS" properly for those functions..

I've attached the three small fixups, Peter/Dave, let me know what's the
best way to redo this.

Thanks,

-- 
Peter Xu
From 7d361c8d61a51ed0df9e1606c3a6f8c306028181 Mon Sep 17 00:00:00 2001
From: Peter Xu <peterx@redhat.com>
Date: Tue, 19 Jul 2022 18:16:40 -0400
Subject: [PATCH 1/3] fixup! tests: Add postcopy tls migration test
Content-type: text/plain

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 tests/qtest/migration-test.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 81780189a8..87dc87ba8b 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1133,6 +1133,7 @@ static void test_postcopy(void)
     test_postcopy_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
 static void test_postcopy_tls_psk(void)
 {
     MigrateCommon args = {
@@ -1142,6 +1143,7 @@ static void test_postcopy_tls_psk(void)
 
     test_postcopy_common(&args);
 }
+#endif
 
 static void test_postcopy_preempt(void)
 {
-- 
2.32.0

From c76945ab7b9a38456f077267ccb51133ef087e35 Mon Sep 17 00:00:00 2001
From: Peter Xu <peterx@redhat.com>
Date: Tue, 19 Jul 2022 18:16:57 -0400
Subject: [PATCH 2/3] fixup! tests: Add postcopy preempt tests
Content-type: text/plain

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 tests/qtest/migration-test.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 87dc87ba8b..490ee71b75 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1154,6 +1154,7 @@ static void test_postcopy_preempt(void)
     test_postcopy_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
 static void test_postcopy_preempt_tls_psk(void)
 {
     MigrateCommon args = {
@@ -1164,6 +1165,7 @@ static void test_postcopy_preempt_tls_psk(void)
 
     test_postcopy_common(&args);
 }
+#endif
 
 static void test_postcopy_recovery_common(MigrateCommon *args)
 {
@@ -1253,6 +1255,7 @@ static void test_postcopy_preempt_recovery(void)
     test_postcopy_recovery_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
 /* This contains preempt+recovery+tls test altogether */
 static void test_postcopy_preempt_all(void)
 {
@@ -1264,6 +1267,7 @@ static void test_postcopy_preempt_all(void)
 
     test_postcopy_recovery_common(&args);
 }
+#endif
 
 static void test_baddest(void)
 {
-- 
2.32.0

From 3f6e48d9cd915378284fe897ddab19112c097f1f Mon Sep 17 00:00:00 2001
From: Peter Xu <peterx@redhat.com>
Date: Tue, 19 Jul 2022 18:17:05 -0400
Subject: [PATCH 3/3] fixup! tests: Add postcopy tls recovery migration test
Content-type: text/plain

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 tests/qtest/migration-test.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 490ee71b75..b0843d35d4 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1236,6 +1236,7 @@ static void test_postcopy_recovery(void)
     test_postcopy_recovery_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
 static void test_postcopy_recovery_tls_psk(void)
 {
     MigrateCommon args = {
@@ -1245,6 +1246,7 @@ static void test_postcopy_recovery_tls_psk(void)
 
     test_postcopy_recovery_common(&args);
 }
+#endif
 
 static void test_postcopy_preempt_recovery(void)
 {
-- 
2.32.0

Re: [PULL 00/29] migration queue
Posted by Dr. David Alan Gilbert 1 year, 8 months ago
* Peter Xu (peterx@redhat.com) wrote:
> On Tue, Jul 19, 2022 at 10:53:33PM +0100, Peter Maydell wrote:
> > On Tue, 19 Jul 2022 at 18:16, Dr. David Alan Gilbert (git)
> > <dgilbert@redhat.com> wrote:
> > >
> > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > >
> > > The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:
> > >
> > >   Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)
> > >
> > > are available in the Git repository at:
> > >
> > >   https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c
> > >
> > > for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:
> > >
> > >   migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)
> > >
> > > ----------------------------------------------------------------
> > > Migration pull 2022-07-19
> > >
> > >   Hyman's dirty page rate limit set
> > >   Ilya's fix for zlib vs migration
> > >   Peter's postcopy-preempt
> > >   Cleanup from Dan
> > >   zero-copy tidy ups from Leo
> > >   multifd doc fix from Juan
> > >
> > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > >
> > > ----------------------------------------------------------------
> > 
> > Fails to build on some configs, eg:
> > https://gitlab.com/qemu-project/qemu/-/jobs/2743059797
> > https://gitlab.com/qemu-project/qemu/-/jobs/2743059743
> > 
> > ../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_tls_psk':
> > ../tests/qtest/migration-test.c:1168:23: error:
> > 'test_migrate_tls_psk_start_match' undeclared (first use in this
> > function)
> > 1168 | .start_hook = test_migrate_tls_psk_start_match,
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c:1168:23: note: each undeclared
> > identifier is reported only once for each function it appears in
> > ../tests/qtest/migration-test.c:1169:24: error:
> > 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> > 1169 | .finish_hook = test_migrate_tls_psk_finish,
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c: In function 'test_postcopy_recovery_tls_psk':
> > ../tests/qtest/migration-test.c:1247:23: error:
> > 'test_migrate_tls_psk_start_match' undeclared (first use in this
> > function)
> > 1247 | .start_hook = test_migrate_tls_psk_start_match,
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c:1248:24: error:
> > 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> > 1248 | .finish_hook = test_migrate_tls_psk_finish,
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c: In function 'test_postcopy_preempt_all':
> > ../tests/qtest/migration-test.c:1268:23: error:
> > 'test_migrate_tls_psk_start_match' undeclared (first use in this
> > function)
> > 1268 | .start_hook = test_migrate_tls_psk_start_match,
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c:1269:24: error:
> > 'test_migrate_tls_psk_finish' undeclared (first use in this function)
> > 1269 | .finish_hook = test_migrate_tls_psk_finish,
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > At top level:
> > ../tests/qtest/migration-test.c:1264:13: error:
> > 'test_postcopy_preempt_all' defined but not used
> > [-Werror=unused-function]
> > 1264 | static void test_postcopy_preempt_all(void)
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c:1244:13: error:
> > 'test_postcopy_recovery_tls_psk' defined but not used
> > [-Werror=unused-function]
> > 1244 | static void test_postcopy_recovery_tls_psk(void)
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > ../tests/qtest/migration-test.c:1164:13: error:
> > 'test_postcopy_preempt_tls_psk' defined but not used
> > [-Werror=unused-function]
> > 1164 | static void test_postcopy_preempt_tls_psk(void)
> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> Sorry my fault.  We'll need to fix the 3 test patches one by one to use "#ifdef
> CONFIG_GNUTLS" properly for those functions..

and mine, I fixed one of those up but missed the others.

> I've attached the three small fixups, Peter/Dave, let me know what's the
> best way to redo this.

I've manually merged them in, and will resend.

Dave

> Thanks,
> 
> -- 
> Peter Xu

> From 7d361c8d61a51ed0df9e1606c3a6f8c306028181 Mon Sep 17 00:00:00 2001
> From: Peter Xu <peterx@redhat.com>
> Date: Tue, 19 Jul 2022 18:16:40 -0400
> Subject: [PATCH 1/3] fixup! tests: Add postcopy tls migration test
> Content-type: text/plain
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  tests/qtest/migration-test.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index 81780189a8..87dc87ba8b 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -1133,6 +1133,7 @@ static void test_postcopy(void)
>      test_postcopy_common(&args);
>  }
>  
> +#ifdef CONFIG_GNUTLS
>  static void test_postcopy_tls_psk(void)
>  {
>      MigrateCommon args = {
> @@ -1142,6 +1143,7 @@ static void test_postcopy_tls_psk(void)
>  
>      test_postcopy_common(&args);
>  }
> +#endif
>  
>  static void test_postcopy_preempt(void)
>  {
> -- 
> 2.32.0
> 

> From c76945ab7b9a38456f077267ccb51133ef087e35 Mon Sep 17 00:00:00 2001
> From: Peter Xu <peterx@redhat.com>
> Date: Tue, 19 Jul 2022 18:16:57 -0400
> Subject: [PATCH 2/3] fixup! tests: Add postcopy preempt tests
> Content-type: text/plain
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  tests/qtest/migration-test.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index 87dc87ba8b..490ee71b75 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -1154,6 +1154,7 @@ static void test_postcopy_preempt(void)
>      test_postcopy_common(&args);
>  }
>  
> +#ifdef CONFIG_GNUTLS
>  static void test_postcopy_preempt_tls_psk(void)
>  {
>      MigrateCommon args = {
> @@ -1164,6 +1165,7 @@ static void test_postcopy_preempt_tls_psk(void)
>  
>      test_postcopy_common(&args);
>  }
> +#endif
>  
>  static void test_postcopy_recovery_common(MigrateCommon *args)
>  {
> @@ -1253,6 +1255,7 @@ static void test_postcopy_preempt_recovery(void)
>      test_postcopy_recovery_common(&args);
>  }
>  
> +#ifdef CONFIG_GNUTLS
>  /* This contains preempt+recovery+tls test altogether */
>  static void test_postcopy_preempt_all(void)
>  {
> @@ -1264,6 +1267,7 @@ static void test_postcopy_preempt_all(void)
>  
>      test_postcopy_recovery_common(&args);
>  }
> +#endif
>  
>  static void test_baddest(void)
>  {
> -- 
> 2.32.0
> 

> From 3f6e48d9cd915378284fe897ddab19112c097f1f Mon Sep 17 00:00:00 2001
> From: Peter Xu <peterx@redhat.com>
> Date: Tue, 19 Jul 2022 18:17:05 -0400
> Subject: [PATCH 3/3] fixup! tests: Add postcopy tls recovery migration test
> Content-type: text/plain
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  tests/qtest/migration-test.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index 490ee71b75..b0843d35d4 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -1236,6 +1236,7 @@ static void test_postcopy_recovery(void)
>      test_postcopy_recovery_common(&args);
>  }
>  
> +#ifdef CONFIG_GNUTLS
>  static void test_postcopy_recovery_tls_psk(void)
>  {
>      MigrateCommon args = {
> @@ -1245,6 +1246,7 @@ static void test_postcopy_recovery_tls_psk(void)
>  
>      test_postcopy_recovery_common(&args);
>  }
> +#endif
>  
>  static void test_postcopy_preempt_recovery(void)
>  {
> -- 
> 2.32.0
> 

-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: [PULL 00/29] migration queue
Posted by Peter Maydell 1 year, 8 months ago
On Tue, 19 Jul 2022 at 18:16, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
>
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:
>
>   Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c
>
> for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:
>
>   migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)
>
> ----------------------------------------------------------------
> Migration pull 2022-07-19
>
>   Hyman's dirty page rate limit set
>   Ilya's fix for zlib vs migration

I'm processing this pullreq, but while I think about it: once
we've got this fix in can we revert the workarounds we put in
our CI configs to set DFLTCC? (ie commit 309df6acb29346f)

thanks
-- PMM
Re: [PULL 00/29] migration queue
Posted by Dr. David Alan Gilbert 1 year, 8 months ago
* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Tue, 19 Jul 2022 at 18:16, Dr. David Alan Gilbert (git)
> <dgilbert@redhat.com> wrote:
> >
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > The following changes since commit da7da9d5e608200ecc0749ff37be246e9cd3314f:
> >
> >   Merge tag 'pull-request-2022-07-19' of https://gitlab.com/thuth/qemu into staging (2022-07-19 13:05:06 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/dagrh/qemu.git tags/pull-migration-20220719c
> >
> > for you to fetch changes up to ec0345c1000b3a57b557da4c2e3f2114dd23903a:
> >
> >   migration: Avoid false-positive on non-supported scenarios for zero-copy-send (2022-07-19 17:33:22 +0100)
> >
> > ----------------------------------------------------------------
> > Migration pull 2022-07-19
> >
> >   Hyman's dirty page rate limit set
> >   Ilya's fix for zlib vs migration
> 
> I'm processing this pullreq, but while I think about it: once
> we've got this fix in can we revert the workarounds we put in
> our CI configs to set DFLTCC? (ie commit 309df6acb29346f)

Good idea, patch coming up, if someone can ack it quickly I'll get it in
with the fix for those build problems.

Dave

> thanks
> -- PMM
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK