Hey,

The cost of switchover is usually not accounted in the migration
algorithm, as the migration algorithm reduces all of it to "pending
bytes" fitting a "threshold" (which represents some available or
proactively-measured link bandwidth) as the rule of thumb to calculate
downtime.

External latencies (OS, or Qemu ones), as well as when VFs are
present, may affect how big or small the switchover may be. Given the wide
range of configurations possible, it is either non exactly determinist or
predictable to have some generic rule to calculate the cost of switchover.

This series is aimed at improving observability what contributes to the
switchover/downtime particularly. The breakdown:

* The first 2 patches move storage of downtime timestamps to its dedicated
data structure, and then we add a couple key places to measure those
timestamps. 

* What we do with those timestamps is the next 2 patches by
calculating the downtime breakdown when asked for the data as well as
adding the tracepointt.

* Finally last patch provides introspection to the
calculated expected-downtime (pending_bytes vs threshold_size) which is
when we decide to switchover, and print that data when available to give
some comparison.

For now, mainly precopy data, and here I added both tracepoints and
QMP stats via query-migrate. Postcopy is still missing.

Thoughts, comments appreciated as usual.

Thanks!
	Joao

Joao Martins (5):
  migration: Store downtime timestamps in an array
  migration: Collect more timestamps during switchover
  migration: Add a tracepoint for the downtime stats
  migration: Provide QMP access to downtime stats
  migration: Print expected-downtime on completion

 qapi/migration.json    | 50 +++++++++++++++++++++++++
 migration/migration.h  |  7 +++-
 migration/migration.c  | 85 ++++++++++++++++++++++++++++++++++++++++--
 migration/savevm.c     |  2 +
 migration/trace-events |  1 +
 5 files changed, 139 insertions(+), 6 deletions(-)

-- 
2.39.3

On Tue, Sep 26, 2023 at 05:18:36PM +0100, Joao Martins wrote:
> For now, mainly precopy data, and here I added both tracepoints and
> QMP stats via query-migrate. Postcopy is still missing.

IIUC many of those will cover postcopy too, but not all?

I think the problem is postcopy didn't update downtime_start, however it
updates MigrationState.downtime, and probably we can start to keep it more
like precopy, e.g., in postcopy_start(), where downtime_start can be
time_at_stop (or it can be more accurate; now it's probably fetching the
timestamp too early).

Basically if we want to expose anything, especially some qapi object, IMHO
we'd better make it work for both pre/post copy because otherwise it'll be
hard for mgmt app to know which qemu supports precopy only, and which
support both (if we'll add that for postcopy too).

Thanks,

-- 
Peter Xu

On 04/10/2023 18:19, Peter Xu wrote:
> On Tue, Sep 26, 2023 at 05:18:36PM +0100, Joao Martins wrote:
>> For now, mainly precopy data, and here I added both tracepoints and
>> QMP stats via query-migrate. Postcopy is still missing.
> 
> IIUC many of those will cover postcopy too, but not all?
> 
> I think the problem is postcopy didn't update downtime_start, however it
> updates MigrationState.downtime, and probably we can start to keep it more
> like precopy, e.g., in postcopy_start(), where downtime_start can be
> time_at_stop (or it can be more accurate; now it's probably fetching the
> timestamp too early).
> 
Good point!

> Basically if we want to expose anything, especially some qapi object, IMHO
> we'd better make it work for both pre/post copy because otherwise it'll be
> hard for mgmt app to know which qemu supports precopy only, and which
> support both (if we'll add that for postcopy too).
> 

Yeap, I totally agree.