Support mlx5_vfio_pci style VFIO drivers (which support live migration) for
cpr-exec and cpr-transfer.
When the VFIO driver is vfio-pci, vfio_migration_query_flags() returns
-ENOTTY, so savevm_vfio_handlers is not registered or involved.
However, VFIO drivers that support live migration, such as mlx5_vfio_pci,
do support vfio_migration_query_flags(). In that case, savevm_vfio_handlers
is registered and may be used to save and load the VFIO migration stream.
This is expected for VFIO live migration, but not for cpr-exec or
cpr-transfer.
For example, the cpr-exec below never "converges".
(qemu) info migrate
Status: active
Time (ms): total=6438, setup=15, exp_down=595047
Remaining: 6.47 MiB
RAM info:
Throughput (Mbps): 0.09
Sizes: pagesize=4 KiB, total=0 B
Transfers: transferred=71.9 KiB, remain=0 B
Channels: precopy=0 B, multifd=0 B, postcopy=0 B, vfio=0 B
Page Types: normal=0, zero=0
Page Rates (pps): transfer=0
Others: dirty_syncs=4033, downtime_bytes=32256
(qemu) info migrate
Status: active
Time (ms): total=7141, setup=15, exp_down=598507
Remaining: 6.47 MiB
RAM info:
Throughput (Mbps): 0.09
Sizes: pagesize=4 KiB, total=0 B
Transfers: transferred=79.7 KiB, remain=0 B
Channels: precopy=0 B, multifd=0 B, postcopy=0 B, vfio=0 B
Page Types: normal=0, zero=0
Page Rates (pps): transfer=0
Others: dirty_syncs=4478, downtime_bytes=35816
(qemu) info migrate
Status: active
Time (ms): total=7682, setup=15, exp_down=595047
Remaining: 6.47 MiB
RAM info:
Throughput (Mbps): 0.09
Sizes: pagesize=4 KiB, total=0 B
Transfers: transferred=85.7 KiB, remain=0 B
Channels: precopy=0 B, multifd=0 B, postcopy=0 B, vfio=0 B
Page Types: normal=0, zero=0
Page Rates (pps): transfer=0
Others: dirty_syncs=4820, downtime_bytes=38552
Even after migration converges with
"migrate_set_parameter downtime-limit 1000000", the target still reports
the following error:
failed (load of migration failed: Invalid argument: Failed to load vmstate version_id: 1, ret: -22)
Skip savevm_vfio_handlers for cpr-exec and cpr-transfer.
Dongli Zhang (2)
vfio/migration: Avoid savevm_vfio_handlers for cpr-exec and cpr-transfer
migration: Avoid inactive SaveVMHandlers in qemu_savevm_state_non_iterable()
hw/vfio/migration.c | 8 ++++++++
migration/savevm.c | 6 +++++-
2 files changed, 13 insertions(+), 1 deletion(-)
base-commit: eea8fe61b8be8f3016e522e6af24924a0266ca95
Tested with mlx5_vfio_pci using cpr-exec and cpr-transfer.
Thank you very much!
Dongli Zhang