diff from v2:
============

1. Fix style issues.
2. Rework patch for fetching job info 
   (save logic to use temporary variable when drop vm lock)
3. Update disk stats when block jobs are canceled.
4. Adress a few more corner cases.

This patch series add disks stats to domain job info(stats) as
well as to migration completed event in case nbd scheme is used.

There is little nuisance with qcow2 disks (which is the main scenario
I guess) tied to the way qemu reports stats for this type of disks.
For example if we have 64G disk filled only to 1G then stats
start from 63G and will grow up to 64G on completion. The same way disk stats
will be reported by this patch.

I guess the better way to express the situation is to say we have 64G 'total',
and have 'processed' field grow from 0G to 1G, like in case of memory
stats. [1] is the example of completed memory stats of empty guest
domain, which show difference between processed and total.

There can be a workaround like getting initial blockjob offset position
as a zero but is is rather ugly and racy and like uses undocumented
behaviour.

[1] memory migration stats example
Memory processed: 3.307 MiB
Memory remaining: 0.000 B
Memory total:     1.032 GiB

The above is applied to qemu 2.6 at least.

Patches that were explicitly ACKed in previous review
(up to style issues) marked with A.

Nikolay Shirokovskiy (16):
  qemu: drop code for VIR_DOMAIN_JOB_BOUNDED and timeRemaining
A qemu: introduce qemu domain job status
A qemu: introduce QEMU_DOMAIN_JOB_STATUS_POSTCOPY
A qemu: drop QEMU_MIGRATION_COMPLETED_UPDATE_STATS
A qemu: drop excessive zero-out in qemuMigrationFetchJobStatus
  qemu: refactor fetching migration stats
A qemu: simplify getting completed job stats
  qemu: fail querying destination migration statistics always
  qemu: start all async job with job status active
  qemu: introduce migrating job status
  qemu: always get job condition on getting job stats
  qemu: migrate: show disks stats on job info requests
  qemu: support getting disks stats during stopping block jobs
  qemu: migation: resolve race on getting job info and stopping block jobs
  qemu: migrate: copy disks stats to completed job
  qemu: migration: don't expose incomplete job as complete

 src/qemu/qemu_blockjob.c         |   1 +
 src/qemu/qemu_domain.c           |  38 +++++--
 src/qemu/qemu_domain.h           |  16 ++-
 src/qemu/qemu_driver.c           |  95 ++++++++--------
 src/qemu/qemu_migration.c        | 236 ++++++++++++++++++++++++++-------------
 src/qemu/qemu_migration.h        |  15 ++-
 src/qemu/qemu_migration_cookie.c |   7 +-
 src/qemu/qemu_monitor.c          |   5 +-
 src/qemu/qemu_monitor.h          |   4 +-
 src/qemu/qemu_monitor_json.c     |   4 +-
 src/qemu/qemu_process.c          |  10 +-
 tests/qemumonitorjsontest.c      |   1 +
 12 files changed, 273 insertions(+), 159 deletions(-)

-- 
1.8.3.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

ping

On 11.04.2017 10:39, Nikolay Shirokovskiy wrote:
> diff from v2:
> ============
> 
> 1. Fix style issues.
> 2. Rework patch for fetching job info 
>    (save logic to use temporary variable when drop vm lock)
> 3. Update disk stats when block jobs are canceled.
> 4. Adress a few more corner cases.
> 
> This patch series add disks stats to domain job info(stats) as
> well as to migration completed event in case nbd scheme is used.
> 
> There is little nuisance with qcow2 disks (which is the main scenario
> I guess) tied to the way qemu reports stats for this type of disks.
> For example if we have 64G disk filled only to 1G then stats
> start from 63G and will grow up to 64G on completion. The same way disk stats
> will be reported by this patch.
> 
> I guess the better way to express the situation is to say we have 64G 'total',
> and have 'processed' field grow from 0G to 1G, like in case of memory
> stats. [1] is the example of completed memory stats of empty guest
> domain, which show difference between processed and total.
> 
> There can be a workaround like getting initial blockjob offset position
> as a zero but is is rather ugly and racy and like uses undocumented
> behaviour.
> 
> [1] memory migration stats example
> Memory processed: 3.307 MiB
> Memory remaining: 0.000 B
> Memory total:     1.032 GiB
> 
> The above is applied to qemu 2.6 at least.
> 
> Patches that were explicitly ACKed in previous review
> (up to style issues) marked with A.
> 
> Nikolay Shirokovskiy (16):
>   qemu: drop code for VIR_DOMAIN_JOB_BOUNDED and timeRemaining
> A qemu: introduce qemu domain job status
> A qemu: introduce QEMU_DOMAIN_JOB_STATUS_POSTCOPY
> A qemu: drop QEMU_MIGRATION_COMPLETED_UPDATE_STATS
> A qemu: drop excessive zero-out in qemuMigrationFetchJobStatus
>   qemu: refactor fetching migration stats
> A qemu: simplify getting completed job stats
>   qemu: fail querying destination migration statistics always
>   qemu: start all async job with job status active
>   qemu: introduce migrating job status
>   qemu: always get job condition on getting job stats
>   qemu: migrate: show disks stats on job info requests
>   qemu: support getting disks stats during stopping block jobs
>   qemu: migation: resolve race on getting job info and stopping block jobs
>   qemu: migrate: copy disks stats to completed job
>   qemu: migration: don't expose incomplete job as complete
> 
>  src/qemu/qemu_blockjob.c         |   1 +
>  src/qemu/qemu_domain.c           |  38 +++++--
>  src/qemu/qemu_domain.h           |  16 ++-
>  src/qemu/qemu_driver.c           |  95 ++++++++--------
>  src/qemu/qemu_migration.c        | 236 ++++++++++++++++++++++++++-------------
>  src/qemu/qemu_migration.h        |  15 ++-
>  src/qemu/qemu_migration_cookie.c |   7 +-
>  src/qemu/qemu_monitor.c          |   5 +-
>  src/qemu/qemu_monitor.h          |   4 +-
>  src/qemu/qemu_monitor_json.c     |   4 +-
>  src/qemu/qemu_process.c          |  10 +-
>  tests/qemumonitorjsontest.c      |   1 +
>  12 files changed, 273 insertions(+), 159 deletions(-)
> 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Fri, Jul 14, 2017 at 09:51:48 +0300, Nikolay Shirokovskiy wrote:
> ping

Oops, I completely forgot about this series. But since it is a few
months old, could you resend the series after rebasing it to current
libvirt? I promise to review them while they are fresh.

Jirka

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On 14.07.2017 12:20, Jiri Denemark wrote:
> On Fri, Jul 14, 2017 at 09:51:48 +0300, Nikolay Shirokovskiy wrote:
>> ping
> 
> Oops, I completely forgot about this series. But since it is a few
> months old, could you resend the series after rebasing it to current
> libvirt? I promise to review them while they are fresh.
> 
> Jirka
> 

Hi, Jiri. 

Unfortunately your email was lost in spam somehow and I would
not even notice it if not Max Nestratov.

I'm going to take a long vacation in a few days thus let's postpone the review
till the moment I'll return back.

Nikolay

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list