[PULL 00/34] Migration 20240311 patches

peterx@redhat.com posted 34 patches 8 months, 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20240311215925.40618-1-peterx@redhat.com
Maintainers: Eduardo Habkost <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Yanan Wang <wangyanan55@huawei.com>, Paolo Bonzini <pbonzini@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Alex Williamson <alex.williamson@redhat.com>, "Cédric Le Goater" <clg@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, David Hildenbrand <david@redhat.com>, Peter Xu <peterx@redhat.com>, Fabiano Rosas <farosas@suse.de>, Hailiang Zhang <zhanghailiang@xfusion.com>, Li Zhijian <lizhijian@fujitsu.com>, Zhang Chen <chen.zhang@intel.com>, Jason Wang <jasowang@redhat.com>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>, Hyman Huang <yong.huang@smartx.com>, Song Gao <gaosong@loongson.cn>, Palmer Dabbelt <palmer@dabbelt.com>, Alistair Francis <alistair.francis@wdc.com>, Bin Meng <bin.meng@windriver.com>, Weiwei Li <liwei1518@gmail.com>, Daniel Henrique Barboza <dbarboza@ventanamicro.com>, Liu Zhiwei <zhiwei_liu@linux.alibaba.com>, Thomas Huth <thuth@redhat.com>, Laurent Vivier <lvivier@redhat.com>
docs/devel/migration/main.rst       |   3 +-
qapi/migration.json                 |  38 +++-
include/hw/qdev-properties-system.h |   4 +
include/migration/client-options.h  |  25 +++
include/migration/misc.h            |  18 +-
include/migration/register.h        | 267 +++++++++++++++++++++++++---
include/qemu/typedefs.h             |   2 -
migration/migration.h               |   7 +-
migration/multifd.h                 |  23 ++-
migration/options.h                 |   7 +-
migration/ram.h                     |   3 +-
hw/core/machine.c                   |   4 +-
hw/core/qdev-properties-system.c    |  10 ++
hw/vfio/common.c                    |  17 +-
hw/vfio/container.c                 |   1 -
hw/vfio/migration.c                 |  24 ++-
hw/virtio/vhost-user.c              |   1 -
hw/virtio/virtio-balloon.c          |   2 -
io/channel-file.c                   |   5 -
migration/colo.c                    |  17 +-
migration/file.c                    |   4 +-
migration/migration-hmp-cmds.c      |   9 +
migration/migration.c               |  67 ++++---
migration/multifd-zero-page.c       |  87 +++++++++
migration/multifd-zlib.c            |  21 ++-
migration/multifd-zstd.c            |  20 ++-
migration/multifd.c                 | 120 ++++++++++---
migration/options.c                 |  32 +++-
migration/qemu-file.c               |   5 +-
migration/ram.c                     |  62 +++++--
migration/rdma.c                    |   2 +-
migration/savevm.c                  |  23 +--
net/colo-compare.c                  |   3 +-
net/vhost-vdpa.c                    |   3 +-
stubs/colo.c                        |   1 -
system/dirtylimit.c                 |  13 +-
system/physmem.c                    | 260 +++++++++++++++++----------
system/qdev-monitor.c               |   1 -
target/loongarch/kvm/kvm.c          |   1 -
target/riscv/kvm/kvm-cpu.c          |   4 +-
tests/qtest/migration-test.c        |  52 ++++++
tests/unit/test-vmstate.c           |   1 -
migration/meson.build               |   1 +
migration/trace-events              |   8 +-
44 files changed, 981 insertions(+), 297 deletions(-)
create mode 100644 include/migration/client-options.h
create mode 100644 migration/multifd-zero-page.c
[PULL 00/34] Migration 20240311 patches
Posted by peterx@redhat.com 8 months, 2 weeks ago
From: Peter Xu <peterx@redhat.com>

The following changes since commit 7489f7f3f81dcb776df8c1b9a9db281fc21bf05f:

  Merge tag 'hw-misc-20240309' of https://github.com/philmd/qemu into staging (2024-03-09 20:12:21 +0000)

are available in the Git repository at:

  https://gitlab.com/peterx/qemu.git tags/migration-20240311-pull-request

for you to fetch changes up to 1815338df00fd0a3fe25085564c6966f74c8f43d:

  migration/multifd: Add new migration test cases for legacy zero page checking. (2024-03-11 16:57:09 -0400)

----------------------------------------------------------------
Migration pull request

- Avihai's fix to allow vmstate iterators to not starve for VFIO
- Maksim's fix on additional check on precopy load error
- Fabiano's fix on fdatasync() hang in mapped-ram
- Jonathan's fix on vring cached access over MMIO regions
- Cedric's cleanup patches 1-4 out of his error report series
- Yu's fix for RDMA migration (which used to be broken even for 8.2)
- Anthony's small cleanup/fix on err message
- Steve's patches on privatize migration.h
- Xiang's patchset to enable zero page detections in multifd threads

----------------------------------------------------------------

Anthony PERARD (1):
  migration: Fix format in error message

Avihai Horon (3):
  migration: Don't serialize devices in qemu_savevm_state_iterate()
  vfio/migration: Refactor vfio_save_state() return value
  vfio/migration: Add a note about migration rate limiting

Cédric Le Goater (4):
  migration: Report error when shutdown fails
  migration: Remove SaveStateHandler and LoadStateHandler typedefs
  migration: Add documentation for SaveVMHandlers
  migration: Do not call PRECOPY_NOTIFY_SETUP notifiers in case of error

Fabiano Rosas (3):
  migration/multifd: Don't fsync when closing QIOChannelFile
  migration/multifd: Allow zero pages in file migration
  migration/multifd: Allow clearing of the file_bmap from multifd

Hao Xiang (5):
  migration/multifd: Add new migration option zero-page-detection.
  migration/multifd: Implement zero page transmission on the multifd
    thread.
  migration/multifd: Implement ram_save_target_page_multifd to handle
    multifd version of MigrationOps::ram_save_target_page.
  migration/multifd: Enable multifd zero page checking by default.
  migration/multifd: Add new migration test cases for legacy zero page
    checking.

Jonathan Cameron (4):
  physmem: Rename addr1 to more informative mr_addr in
    flatview_read/write() and similar
  physmem: Reduce local variable scope in flatview_read/write_continue()
  physmem: Factor out body of flatview_read/write_continue() loop
  physmem: Fix wrong address in large
    address_space_read/write_cached_slow()

Maksim Davydov (1):
  migration/ram: add additional check

Steve Sistare (12):
  migration: export fewer options
  migration: remove migration.h references
  migration: export migration_is_setup_or_active
  migration: export migration_is_active
  migration: export migration_is_running
  migration: export vcpu_dirty_limit_period
  migration: migration_thread_is_self
  migration: migration_is_device
  migration: migration_file_set_error
  migration: privatize colo interfaces
  migration: delete unused accessors
  migration: purge MigrationState from public interface

Yu Zhang (1):
  migration/rdma: Fix a memory issue for migration

 docs/devel/migration/main.rst       |   3 +-
 qapi/migration.json                 |  38 +++-
 include/hw/qdev-properties-system.h |   4 +
 include/migration/client-options.h  |  25 +++
 include/migration/misc.h            |  18 +-
 include/migration/register.h        | 267 +++++++++++++++++++++++++---
 include/qemu/typedefs.h             |   2 -
 migration/migration.h               |   7 +-
 migration/multifd.h                 |  23 ++-
 migration/options.h                 |   7 +-
 migration/ram.h                     |   3 +-
 hw/core/machine.c                   |   4 +-
 hw/core/qdev-properties-system.c    |  10 ++
 hw/vfio/common.c                    |  17 +-
 hw/vfio/container.c                 |   1 -
 hw/vfio/migration.c                 |  24 ++-
 hw/virtio/vhost-user.c              |   1 -
 hw/virtio/virtio-balloon.c          |   2 -
 io/channel-file.c                   |   5 -
 migration/colo.c                    |  17 +-
 migration/file.c                    |   4 +-
 migration/migration-hmp-cmds.c      |   9 +
 migration/migration.c               |  67 ++++---
 migration/multifd-zero-page.c       |  87 +++++++++
 migration/multifd-zlib.c            |  21 ++-
 migration/multifd-zstd.c            |  20 ++-
 migration/multifd.c                 | 120 ++++++++++---
 migration/options.c                 |  32 +++-
 migration/qemu-file.c               |   5 +-
 migration/ram.c                     |  62 +++++--
 migration/rdma.c                    |   2 +-
 migration/savevm.c                  |  23 +--
 net/colo-compare.c                  |   3 +-
 net/vhost-vdpa.c                    |   3 +-
 stubs/colo.c                        |   1 -
 system/dirtylimit.c                 |  13 +-
 system/physmem.c                    | 260 +++++++++++++++++----------
 system/qdev-monitor.c               |   1 -
 target/loongarch/kvm/kvm.c          |   1 -
 target/riscv/kvm/kvm-cpu.c          |   4 +-
 tests/qtest/migration-test.c        |  52 ++++++
 tests/unit/test-vmstate.c           |   1 -
 migration/meson.build               |   1 +
 migration/trace-events              |   8 +-
 44 files changed, 981 insertions(+), 297 deletions(-)
 create mode 100644 include/migration/client-options.h
 create mode 100644 migration/multifd-zero-page.c

-- 
2.44.0


Re: [PULL 00/34] Migration 20240311 patches
Posted by Peter Maydell 8 months, 2 weeks ago
On Mon, 11 Mar 2024 at 21:59, <peterx@redhat.com> wrote:
>
> From: Peter Xu <peterx@redhat.com>
>
> The following changes since commit 7489f7f3f81dcb776df8c1b9a9db281fc21bf05f:
>
>   Merge tag 'hw-misc-20240309' of https://github.com/philmd/qemu into staging (2024-03-09 20:12:21 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/peterx/qemu.git tags/migration-20240311-pull-request
>
> for you to fetch changes up to 1815338df00fd0a3fe25085564c6966f74c8f43d:
>
>   migration/multifd: Add new migration test cases for legacy zero page checking. (2024-03-11 16:57:09 -0400)
>
> ----------------------------------------------------------------
> Migration pull request
>
> - Avihai's fix to allow vmstate iterators to not starve for VFIO
> - Maksim's fix on additional check on precopy load error
> - Fabiano's fix on fdatasync() hang in mapped-ram
> - Jonathan's fix on vring cached access over MMIO regions
> - Cedric's cleanup patches 1-4 out of his error report series
> - Yu's fix for RDMA migration (which used to be broken even for 8.2)
> - Anthony's small cleanup/fix on err message
> - Steve's patches on privatize migration.h
> - Xiang's patchset to enable zero page detections in multifd threads
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/9.0
for any user-visible changes.

-- PMM