[PATCH v7 0/7] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate

Hao Wang posted 7 patches 3 years, 1 month ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20210316123251.511485-1-wanghao232@huawei.com
docs/manpages/virsh.rst          |  33 +++++++++-
include/libvirt/libvirt-domain.h |  23 +++++++
src/driver-hypervisor.h          |   6 ++
src/libvirt-domain.c             |  59 +++++++++++++++++
src/libvirt_public.syms          |   5 ++
src/qemu/qemu_driver.c           | 110 +++++++++++++++++++++++++++++++
src/qemu/qemu_monitor.c          |  24 +++++++
src/qemu/qemu_monitor.h          |  19 ++++++
src/qemu/qemu_monitor_json.c     | 101 ++++++++++++++++++++++++++++
src/qemu/qemu_monitor_json.h     |   8 +++
src/remote/remote_driver.c       |   1 +
src/remote/remote_protocol.x     |  14 +++-
src/remote_protocol-structs      |   6 ++
tools/virsh-domain-monitor.c     |   7 ++
tools/virsh-domain.c             |  58 ++++++++++++++++
15 files changed, 471 insertions(+), 3 deletions(-)
[PATCH v7 0/7] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate
Posted by Hao Wang 3 years, 1 month ago
V6 -> V7:
add virDomainDirtyRateStatus enum
define calc_status as int
split 5/5 in v6 into two parts:
    6/7 in v7: extend dirtyrate statistics for domGetStats
    7/7 in v7: extend corresponding virsh command

V5 -> V6:
split DomainGetDirtyRateInfo(domdirtyrate) API into two parts:
    1. DomainStartDirtyRateCalc(domdirtyrate-calc) for starting dirty rate
       calculation;
    2. qemuDomainGetStatsDirtyRate(domstats --dirtyrate) for querying dirty
       rate infomation.

V4 -> V5:
squash 1/7 and bits of 5/7 and 6/7 into 2/7 in v4 (to be 1/5 in v5)
squash left of 5/7 into 4/7 in v4 (to be 3/5 in v5)
add VIR_DOMAIN_DIRTYRATE_DEFAULT flag
remove redundant error report
rename virsh api to "domdirtyrate"
use vshTablePtr for virsh api output
add description in docs/manpages/virsh.rst
other format optimize

V3 -> V4:
define flags to unsigned int
fix some compile warnings

V2 -> V3:
reorganize patchset to fix compile warning

V1 -> V2:
replace QEMU_JOB_ASYNC with QEMU_JOB_QUERY


Sometimes domain's memory dirty rate is expected by user in order to
decide whether it's proper to be migrated out or not.

We have already completed the QEMU part of the capability:
https://patchew.org/QEMU/1600237327-33618-1-git-send-email-zhengchuan@huawei.com/
And this serial of patches introduce the corresponding LIBVIRT part:

1. Calculating
Introduce a new API DomainStartDirtyRateCalc and corresponding virsh api
(domdirtyrate-calc) for starting dirty rate calculation by calling qmp
'calc-dirty-rate'.

# virsh domdirtyrate-calc <domain> [--seconds <sec>]

2. Querying
Introduce command 'virsh domstats --dirtyrate' for reporting memory
dirty rate infomation by calling qmp 'query-dirty-rate'.

The info is listed as:
Domain: 'vm0'
  dirtyrate.calc_status=2
  dirtyrate.calc_start_time=1534523
  dirtyrate.calc_period=1
  dirtyrate.megabytes_per_second=5


Hao Wang (7):
  migration/dirtyrate: Introduce virDomainStartDirtyRateCalc API
  migration/dirtyrate: Implement qemuDomainStartDirtyRateCalc
  migration/dirtyrate: Introduce domdirtyrate-calc virsh api
  migration/dirtyrate: Introduce virDomainDirtyRateStatus enum
  migration/dirtyrate: Implement qemuMonitorQueryDirtyRate
  migration/dirtyrate: Extend dirtyrate statistics for domGetStats
  migration/dirtyrate: Introduce command 'virsh domstats --dirtyrate'

 docs/manpages/virsh.rst          |  33 +++++++++-
 include/libvirt/libvirt-domain.h |  23 +++++++
 src/driver-hypervisor.h          |   6 ++
 src/libvirt-domain.c             |  59 +++++++++++++++++
 src/libvirt_public.syms          |   5 ++
 src/qemu/qemu_driver.c           | 110 +++++++++++++++++++++++++++++++
 src/qemu/qemu_monitor.c          |  24 +++++++
 src/qemu/qemu_monitor.h          |  19 ++++++
 src/qemu/qemu_monitor_json.c     | 101 ++++++++++++++++++++++++++++
 src/qemu/qemu_monitor_json.h     |   8 +++
 src/remote/remote_driver.c       |   1 +
 src/remote/remote_protocol.x     |  14 +++-
 src/remote_protocol-structs      |   6 ++
 tools/virsh-domain-monitor.c     |   7 ++
 tools/virsh-domain.c             |  58 ++++++++++++++++
 15 files changed, 471 insertions(+), 3 deletions(-)

-- 
2.23.0


Re: [PATCH v7 0/7] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate
Posted by Michal Privoznik 3 years, 1 month ago
On 3/16/21 1:32 PM, Hao Wang wrote:
> V6 -> V7:
> add virDomainDirtyRateStatus enum
> define calc_status as int
> split 5/5 in v6 into two parts:
>      6/7 in v7: extend dirtyrate statistics for domGetStats
>      7/7 in v7: extend corresponding virsh command
> 
> V5 -> V6:
> split DomainGetDirtyRateInfo(domdirtyrate) API into two parts:
>      1. DomainStartDirtyRateCalc(domdirtyrate-calc) for starting dirty rate
>         calculation;
>      2. qemuDomainGetStatsDirtyRate(domstats --dirtyrate) for querying dirty
>         rate infomation.
> 
> V4 -> V5:
> squash 1/7 and bits of 5/7 and 6/7 into 2/7 in v4 (to be 1/5 in v5)
> squash left of 5/7 into 4/7 in v4 (to be 3/5 in v5)
> add VIR_DOMAIN_DIRTYRATE_DEFAULT flag
> remove redundant error report
> rename virsh api to "domdirtyrate"
> use vshTablePtr for virsh api output
> add description in docs/manpages/virsh.rst
> other format optimize
> 
> V3 -> V4:
> define flags to unsigned int
> fix some compile warnings
> 
> V2 -> V3:
> reorganize patchset to fix compile warning
> 
> V1 -> V2:
> replace QEMU_JOB_ASYNC with QEMU_JOB_QUERY
> 
> 
> Sometimes domain's memory dirty rate is expected by user in order to
> decide whether it's proper to be migrated out or not.
> 
> We have already completed the QEMU part of the capability:
> https://patchew.org/QEMU/1600237327-33618-1-git-send-email-zhengchuan@huawei.com/
> And this serial of patches introduce the corresponding LIBVIRT part:
> 
> 1. Calculating
> Introduce a new API DomainStartDirtyRateCalc and corresponding virsh api
> (domdirtyrate-calc) for starting dirty rate calculation by calling qmp
> 'calc-dirty-rate'.
> 
> # virsh domdirtyrate-calc <domain> [--seconds <sec>]
> 
> 2. Querying
> Introduce command 'virsh domstats --dirtyrate' for reporting memory
> dirty rate infomation by calling qmp 'query-dirty-rate'.
> 
> The info is listed as:
> Domain: 'vm0'
>    dirtyrate.calc_status=2
>    dirtyrate.calc_start_time=1534523
>    dirtyrate.calc_period=1
>    dirtyrate.megabytes_per_second=5
> 
> 
> Hao Wang (7):
>    migration/dirtyrate: Introduce virDomainStartDirtyRateCalc API
>    migration/dirtyrate: Implement qemuDomainStartDirtyRateCalc
>    migration/dirtyrate: Introduce domdirtyrate-calc virsh api
>    migration/dirtyrate: Introduce virDomainDirtyRateStatus enum
>    migration/dirtyrate: Implement qemuMonitorQueryDirtyRate
>    migration/dirtyrate: Extend dirtyrate statistics for domGetStats
>    migration/dirtyrate: Introduce command 'virsh domstats --dirtyrate'
> 
>   docs/manpages/virsh.rst          |  33 +++++++++-
>   include/libvirt/libvirt-domain.h |  23 +++++++
>   src/driver-hypervisor.h          |   6 ++
>   src/libvirt-domain.c             |  59 +++++++++++++++++
>   src/libvirt_public.syms          |   5 ++
>   src/qemu/qemu_driver.c           | 110 +++++++++++++++++++++++++++++++
>   src/qemu/qemu_monitor.c          |  24 +++++++
>   src/qemu/qemu_monitor.h          |  19 ++++++
>   src/qemu/qemu_monitor_json.c     | 101 ++++++++++++++++++++++++++++
>   src/qemu/qemu_monitor_json.h     |   8 +++
>   src/remote/remote_driver.c       |   1 +
>   src/remote/remote_protocol.x     |  14 +++-
>   src/remote_protocol-structs      |   6 ++
>   tools/virsh-domain-monitor.c     |   7 ++
>   tools/virsh-domain.c             |  58 ++++++++++++++++
>   15 files changed, 471 insertions(+), 3 deletions(-)
> 

Looks good to me.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>

Will push shortly. Can you please post a follow up patch that adds 
NEWS.rst entry? I think this deserves to be mention in release notes.

Michal

Re: [PATCH v7 0/7] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate
Posted by Hao Wang 3 years, 1 month ago
Sure. I'll post NEWS.rst information soon, and really appreciate your help!

Best Regards,
Hao

On 2021/3/18 15:49, Michal Privoznik wrote:
> On 3/16/21 1:32 PM, Hao Wang wrote:
>> Hao Wang (7):
>>    migration/dirtyrate: Introduce virDomainStartDirtyRateCalc API
>>    migration/dirtyrate: Implement qemuDomainStartDirtyRateCalc
>>    migration/dirtyrate: Introduce domdirtyrate-calc virsh api
>>    migration/dirtyrate: Introduce virDomainDirtyRateStatus enum
>>    migration/dirtyrate: Implement qemuMonitorQueryDirtyRate
>>    migration/dirtyrate: Extend dirtyrate statistics for domGetStats
>>    migration/dirtyrate: Introduce command 'virsh domstats --dirtyrate'
>>
>>   docs/manpages/virsh.rst          |  33 +++++++++-
>>   include/libvirt/libvirt-domain.h |  23 +++++++
>>   src/driver-hypervisor.h          |   6 ++
>>   src/libvirt-domain.c             |  59 +++++++++++++++++
>>   src/libvirt_public.syms          |   5 ++
>>   src/qemu/qemu_driver.c           | 110 +++++++++++++++++++++++++++++++
>>   src/qemu/qemu_monitor.c          |  24 +++++++
>>   src/qemu/qemu_monitor.h          |  19 +++++
>>   src/qemu/qemu_monitor_json.c     | 101 ++++++++++++++++++++++++++++
>>   src/qemu/qemu_monitor_json.h     |   8 +++
>>   src/remote/remote_driver.c       |   1 +
>>   src/remote/remote_protocol.x     |  14 +++-
>>   src/remote_protocol-structs      |   6 ++
>>   tools/virsh-domain-monitor.c     |   7 ++
>>   tools/virsh-domain.c             |  58 ++++++++++++++++
>>   15 files changed, 471 insertions(+), 3 deletions(-)
>>
> 
> Looks good to me.
> 
> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
> 
> Will push shortly. Can you please post a follow up patch that adds NEWS.rst entry? I think this deserves to be mention in release notes.
> 
> Michal
> 
> .