[PULL 00/18] Migration 20241030 patches

Peter Xu posted 18 patches 3 weeks, 2 days ago
There is a newer version of this series
docs/about/deprecated.rst                    |   8 ++
qapi/migration.json                          |   7 +-
include/migration/misc.h                     |   9 +-
{include/sysemu => migration}/cpu-throttle.h |  14 ++
migration/migration.h                        |  19 +++
migration/ram.h                              |   1 +
accel/tcg/icount-common.c                    |   1 -
hw/vfio/common.c                             |   2 +-
hw/virtio/virtio-mem.c                       |   2 +-
migration/colo.c                             |   3 +-
{system => migration}/cpu-throttle.c         |  72 +++++++++-
migration/dirtyrate.c                        |  11 +-
migration/migration.c                        | 130 ++++++++-----------
migration/multifd.c                          |   8 +-
migration/postcopy-ram.c                     |   6 +-
migration/ram.c                              |  21 +--
migration/savevm.c                           |   3 +-
migration/vmstate.c                          |  13 +-
net/vhost-vdpa.c                             |   3 +-
system/cpu-timers.c                          |   3 -
system/qdev-monitor.c                        |   4 +-
tests/qtest/migration-test.c                 |  32 +++++
migration/meson.build                        |   1 +
migration/trace-events                       |   5 +
system/meson.build                           |   1 -
system/trace-events                          |   3 -
26 files changed, 257 insertions(+), 125 deletions(-)
rename {include/sysemu => migration}/cpu-throttle.h (87%)
rename {system => migration}/cpu-throttle.c (64%)
[PULL 00/18] Migration 20241030 patches
Posted by Peter Xu 3 weeks, 2 days ago
The following changes since commit cc5adbbd50d81555b8eb73602ec16fde40b55be4:

  Merge tag 'pull-tpm-2024-10-18-1' of https://github.com/stefanberger/qemu-tpm into staging (2024-10-18 15:45:02 +0100)

are available in the Git repository at:

  https://gitlab.com/peterx/qemu.git tags/migration-20241030-pull-request

for you to fetch changes up to 53a60118d2654dd8e595e61f4e767ff747fd0b69:

  migration/multifd: Zero p->flags before starting filling a packet (2024-10-30 11:32:41 -0400)

----------------------------------------------------------------
Migration pull request for softfreeze

NOTE: checkpatch.pl could report a false positive on this branch:

  WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
  #21:
   {include/sysemu => migration}/cpu-throttle.h | 0

That's covered by "F: migration/" entry.

Changelog:

- Peter's cleanup patch on migrate_fd_cleanup()
- Peter's cleanup patch to introduce thread name macros
- Hanna's error path fix for vmstate subsection save()s
- Hyman's auto converge enhancement on background dirty sync
- Peter's additional tracepoints for save state entries
- Thomas's build fix for OpenBSD in dirtyrate.c
- Peter's deprecation of query-migrationthreads command
- Peter's cleanup/fixes from the "export misc.h" series
- Maciej's two small patches from multifd+vfio series

----------------------------------------------------------------

Hanna Czenczek (1):
  migration: Ensure vmstate_save() sets errp

Hyman Huang (6):
  accel/tcg/icount-common: Remove the reference to the unused header
    file
  migration: Stop CPU throttling conditionally
  migration: Move cpu-throttole.c from system to migration
  migration: Remove "rs" parameter in migration_bitmap_sync_precopy
  migration: Support periodic RAMBlock dirty bitmap sync
  tests/migration: Add case for periodic ramblock dirty sync

Maciej S. Szmigiero (2):
  migration/ram: Add load start trace event
  migration/multifd: Zero p->flags before starting filling a packet

Peter Xu (8):
  migration: Cleanup migrate_fd_cleanup() on accessing to_dst_file
  migration: Put thread names together with macros
  migration: Deprecate query-migrationthreads command
  migration: Take migration object refcount earlier for threads
  migration: Unexport dirty_bitmap_mig_init()
  migration: Unexport ram_mig_init()
  migration: Drop migration_is_setup_or_active()
  migration: Drop migration_is_idle()

Thomas Huth (1):
  migration/dirtyrate: Silence warning about strcpy() on OpenBSD

 docs/about/deprecated.rst                    |   8 ++
 qapi/migration.json                          |   7 +-
 include/migration/misc.h                     |   9 +-
 {include/sysemu => migration}/cpu-throttle.h |  14 ++
 migration/migration.h                        |  19 +++
 migration/ram.h                              |   1 +
 accel/tcg/icount-common.c                    |   1 -
 hw/vfio/common.c                             |   2 +-
 hw/virtio/virtio-mem.c                       |   2 +-
 migration/colo.c                             |   3 +-
 {system => migration}/cpu-throttle.c         |  72 +++++++++-
 migration/dirtyrate.c                        |  11 +-
 migration/migration.c                        | 130 ++++++++-----------
 migration/multifd.c                          |   8 +-
 migration/postcopy-ram.c                     |   6 +-
 migration/ram.c                              |  21 +--
 migration/savevm.c                           |   3 +-
 migration/vmstate.c                          |  13 +-
 net/vhost-vdpa.c                             |   3 +-
 system/cpu-timers.c                          |   3 -
 system/qdev-monitor.c                        |   4 +-
 tests/qtest/migration-test.c                 |  32 +++++
 migration/meson.build                        |   1 +
 migration/trace-events                       |   5 +
 system/meson.build                           |   1 -
 system/trace-events                          |   3 -
 26 files changed, 257 insertions(+), 125 deletions(-)
 rename {include/sysemu => migration}/cpu-throttle.h (87%)
 rename {system => migration}/cpu-throttle.c (64%)

-- 
2.45.0
Re: [PULL 00/18] Migration 20241030 patches
Posted by Peter Maydell 3 weeks, 2 days ago
On Wed, 30 Oct 2024 at 15:57, Peter Xu <peterx@redhat.com> wrote:
>
> The following changes since commit cc5adbbd50d81555b8eb73602ec16fde40b55be4:
>
>   Merge tag 'pull-tpm-2024-10-18-1' of https://github.com/stefanberger/qemu-tpm into staging (2024-10-18 15:45:02 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/peterx/qemu.git tags/migration-20241030-pull-request
>
> for you to fetch changes up to 53a60118d2654dd8e595e61f4e767ff747fd0b69:
>
>   migration/multifd: Zero p->flags before starting filling a packet (2024-10-30 11:32:41 -0400)
>
> ----------------------------------------------------------------
> Migration pull request for softfreeze
>
> NOTE: checkpatch.pl could report a false positive on this branch:
>
>   WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
>   #21:
>    {include/sysemu => migration}/cpu-throttle.h | 0
>
> That's covered by "F: migration/" entry.
>
> Changelog:
>
> - Peter's cleanup patch on migrate_fd_cleanup()
> - Peter's cleanup patch to introduce thread name macros
> - Hanna's error path fix for vmstate subsection save()s
> - Hyman's auto converge enhancement on background dirty sync
> - Peter's additional tracepoints for save state entries
> - Thomas's build fix for OpenBSD in dirtyrate.c
> - Peter's deprecation of query-migrationthreads command
> - Peter's cleanup/fixes from the "export misc.h" series
> - Maciej's two small patches from multifd+vfio series


>  {include/sysemu => migration}/cpu-throttle.h |  14 ++

Hi; this fails to build on macos:

https://gitlab.com/qemu-project/qemu/-/jobs/8237753019

../ui/cocoa.m:40:10: fatal error: 'sysemu/cpu-throttle.h' file not found
    40 | #include "sysemu/cpu-throttle.h"
       |          ^~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

thanks
-- PMM
Re: [PULL 00/18] Migration 20241030 patches
Posted by Peter Xu 3 weeks, 2 days ago
On Thu, Oct 31, 2024 at 01:28:31PM +0000, Peter Maydell wrote:
> On Wed, 30 Oct 2024 at 15:57, Peter Xu <peterx@redhat.com> wrote:
> >
> > The following changes since commit cc5adbbd50d81555b8eb73602ec16fde40b55be4:
> >
> >   Merge tag 'pull-tpm-2024-10-18-1' of https://github.com/stefanberger/qemu-tpm into staging (2024-10-18 15:45:02 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/peterx/qemu.git tags/migration-20241030-pull-request
> >
> > for you to fetch changes up to 53a60118d2654dd8e595e61f4e767ff747fd0b69:
> >
> >   migration/multifd: Zero p->flags before starting filling a packet (2024-10-30 11:32:41 -0400)
> >
> > ----------------------------------------------------------------
> > Migration pull request for softfreeze
> >
> > NOTE: checkpatch.pl could report a false positive on this branch:
> >
> >   WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> >   #21:
> >    {include/sysemu => migration}/cpu-throttle.h | 0
> >
> > That's covered by "F: migration/" entry.
> >
> > Changelog:
> >
> > - Peter's cleanup patch on migrate_fd_cleanup()
> > - Peter's cleanup patch to introduce thread name macros
> > - Hanna's error path fix for vmstate subsection save()s
> > - Hyman's auto converge enhancement on background dirty sync
> > - Peter's additional tracepoints for save state entries
> > - Thomas's build fix for OpenBSD in dirtyrate.c
> > - Peter's deprecation of query-migrationthreads command
> > - Peter's cleanup/fixes from the "export misc.h" series
> > - Maciej's two small patches from multifd+vfio series
> 
> 
> >  {include/sysemu => migration}/cpu-throttle.h |  14 ++
> 
> Hi; this fails to build on macos:
> 
> https://gitlab.com/qemu-project/qemu/-/jobs/8237753019
> 
> ../ui/cocoa.m:40:10: fatal error: 'sysemu/cpu-throttle.h' file not found
>     40 | #include "sysemu/cpu-throttle.h"
>        |          ^~~~~~~~~~~~~~~~~~~~~~~
> 1 error generated.

Errr, cocoa.m used cpu-throttle for Speed feature.. so it needs it to be
exported.

Luckily cocoa.m is only compiled in system_ss.  The simplest fix so far is
to only move cpu-throttle.c, not cpu-throttle.h yet in the commit:

"migration: Move cpu-throttole.c from system to migration"

Fixup patch to be squashed:

===8<===
From 68515db81e28832cbd24b1cdbc12aeb618c9de54 Mon Sep 17 00:00:00 2001
From: Peter Xu <peterx@redhat.com>
Date: Thu, 31 Oct 2024 10:37:29 -0400
Subject: [PATCH] fixup! migration: Move cpu-throttole.c from system to
 migration

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 {migration => include/sysemu}/cpu-throttle.h | 0
 migration/cpu-throttle.c                     | 2 +-
 migration/migration.c                        | 2 +-
 migration/ram.c                              | 2 +-
 4 files changed, 3 insertions(+), 3 deletions(-)
 rename {migration => include/sysemu}/cpu-throttle.h (100%)

diff --git a/migration/cpu-throttle.h b/include/sysemu/cpu-throttle.h
similarity index 100%
rename from migration/cpu-throttle.h
rename to include/sysemu/cpu-throttle.h
diff --git a/migration/cpu-throttle.c b/migration/cpu-throttle.c
index 3df287d8d3..5179019e33 100644
--- a/migration/cpu-throttle.c
+++ b/migration/cpu-throttle.c
@@ -27,7 +27,7 @@
 #include "hw/core/cpu.h"
 #include "qemu/main-loop.h"
 #include "sysemu/cpus.h"
-#include "cpu-throttle.h"
+#include "sysemu/cpu-throttle.h"
 #include "migration.h"
 #include "migration-stats.h"
 #include "trace.h"
diff --git a/migration/migration.c b/migration/migration.c
index 04d7e67897..aedf7f0751 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -24,7 +24,7 @@
 #include "socket.h"
 #include "sysemu/runstate.h"
 #include "sysemu/sysemu.h"
-#include "cpu-throttle.h"
+#include "sysemu/cpu-throttle.h"
 #include "rdma.h"
 #include "ram.h"
 #include "migration/global_state.h"
diff --git a/migration/ram.c b/migration/ram.c
index 12031df4e5..05ff9eb328 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -52,7 +52,7 @@
 #include "exec/target_page.h"
 #include "qemu/rcu_queue.h"
 #include "migration/colo.h"
-#include "cpu-throttle.h"
+#include "sysemu/cpu-throttle.h"
 #include "savevm.h"
 #include "qemu/iov.h"
 #include "multifd.h"
-- 
2.45.0
===8<===

Hyman/Fabiano, feel free to comment if there's any objections.  I'll wait
for 1 day for that before I send the v2 pull with the planned fix.

Thanks,

-- 
Peter Xu
Re: [PULL 00/18] Migration 20241030 patches
Posted by Peter Maydell 3 weeks, 2 days ago
On Thu, 31 Oct 2024 at 14:50, Peter Xu <peterx@redhat.com> wrote:
>
> On Thu, Oct 31, 2024 at 01:28:31PM +0000, Peter Maydell wrote:
> > Hi; this fails to build on macos:
> >
> > https://gitlab.com/qemu-project/qemu/-/jobs/8237753019
> >
> > ../ui/cocoa.m:40:10: fatal error: 'sysemu/cpu-throttle.h' file not found
> >     40 | #include "sysemu/cpu-throttle.h"
> >        |          ^~~~~~~~~~~~~~~~~~~~~~~
> > 1 error generated.
>
> Errr, cocoa.m used cpu-throttle for Speed feature.. so it needs it to be
> exported.
>
> Luckily cocoa.m is only compiled in system_ss.  The simplest fix so far is
> to only move cpu-throttle.c, not cpu-throttle.h yet in the commit:
>
> "migration: Move cpu-throttole.c from system to migration"
>
> Fixup patch to be squashed:
>
> ===8<===
> From 68515db81e28832cbd24b1cdbc12aeb618c9de54 Mon Sep 17 00:00:00 2001
> From: Peter Xu <peterx@redhat.com>
> Date: Thu, 31 Oct 2024 10:37:29 -0400
> Subject: [PATCH] fixup! migration: Move cpu-throttole.c from system to
>  migration

If you're fixing up that commit anyway you could also fix
the typo in the commit message: s/throttole/throttle/.

-- PMM
Re: [PULL 00/18] Migration 20241030 patches
Posted by Peter Xu 3 weeks, 1 day ago
On Thu, Oct 31, 2024 at 02:52:28PM +0000, Peter Maydell wrote:
> On Thu, 31 Oct 2024 at 14:50, Peter Xu <peterx@redhat.com> wrote:
> >
> > On Thu, Oct 31, 2024 at 01:28:31PM +0000, Peter Maydell wrote:
> > > Hi; this fails to build on macos:
> > >
> > > https://gitlab.com/qemu-project/qemu/-/jobs/8237753019
> > >
> > > ../ui/cocoa.m:40:10: fatal error: 'sysemu/cpu-throttle.h' file not found
> > >     40 | #include "sysemu/cpu-throttle.h"
> > >        |          ^~~~~~~~~~~~~~~~~~~~~~~
> > > 1 error generated.
> >
> > Errr, cocoa.m used cpu-throttle for Speed feature.. so it needs it to be
> > exported.
> >
> > Luckily cocoa.m is only compiled in system_ss.  The simplest fix so far is
> > to only move cpu-throttle.c, not cpu-throttle.h yet in the commit:
> >
> > "migration: Move cpu-throttole.c from system to migration"
> >
> > Fixup patch to be squashed:
> >
> > ===8<===
> > From 68515db81e28832cbd24b1cdbc12aeb618c9de54 Mon Sep 17 00:00:00 2001
> > From: Peter Xu <peterx@redhat.com>
> > Date: Thu, 31 Oct 2024 10:37:29 -0400
> > Subject: [PATCH] fixup! migration: Move cpu-throttole.c from system to
> >  migration
> 
> If you're fixing up that commit anyway you could also fix
> the typo in the commit message: s/throttole/throttle/.

Will do.

-- 
Peter Xu