docs/formatdomain.rst | 7 +- docs/manpages/virsh.rst | 33 +++- include/libvirt/libvirt-domain.h | 5 + src/conf/domain_conf.c | 26 +++ src/conf/domain_conf.h | 8 + src/conf/domain_validate.c | 33 ++++ src/conf/domain_validate.h | 2 + src/conf/schemas/domaincommon.rng | 5 + src/driver-hypervisor.h | 7 + src/libvirt-domain.c | 68 +++++++ src/libvirt_private.syms | 1 + src/libvirt_public.syms | 5 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_driver.c | 181 ++++++++++++++++++ src/qemu/qemu_monitor.c | 25 +++ src/qemu/qemu_monitor.h | 22 +++ src/qemu/qemu_monitor_json.c | 107 +++++++++++ src/qemu/qemu_monitor_json.h | 9 + src/qemu/qemu_process.c | 44 +++++ src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 17 +- src/remote_protocol-structs | 7 + .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 + .../caps_7.1.0_x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 + .../caps_7.2.0_x86_64+hvf.xml | 1 + .../caps_7.2.0_x86_64.xml | 1 + .../caps_8.0.0_riscv64.xml | 1 + .../caps_8.0.0_x86_64.xml | 1 + .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 + .../caps_8.1.0_x86_64.xml | 1 + tools/virsh-domain-monitor.c | 7 + tools/virsh-domain.c | 109 +++++++++++ 34 files changed, 737 insertions(+), 4 deletions(-)
Hi, This is the latest version for the series, comparing with version
1, there are some key modifications has made inspired and
suggested by Peter, see as follows:
1. Introduce XML for dirty limit persistent configuration
2. Merge the cancel API into the set API
3. Extend the domstats/virDomainListGetStats API for dirty limit
information query
4. Introduce the virDomainModificationImpact flags to control the
behavior of the API
5. Enrich the comments and docs about the feature and API
The patch set introduce the new API virDomainSetVcpuDirtyLimit to
allow upper Apps to set upper limits of dirty page rate for virtual
CPUs,
the corresponding virsh API as follows:
# limit-dirty-page-rate <domain> <rate> [--vcpu <number>] \
[--config] [--live] [--current]
We put the dirty limit persistent info with the "vcpus" element in
domain XML and
extend dirtylimit statistics for domGetStats:
<domain>
...
<vcpu current='2'>3</vcpu>
<vcpus>
<vcpu id='0' hotpluggable='no' dirty_limit='10' order='1'.../>
<vcpu id='1' hotpluggable='yes' dirty_limit='10' order='2'.../>
</vcpus>
...
If --vcpu option is not passed in the virsh command, set all virtual
CPUs;
if rate is set to zero, cancel the upper limit.
Examples:
To set the dirty page rate upper limit 10 MB/s for all virtual CPUs in
c81_node1, use:
[root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 --rate 10
--live
Set dirty page rate limit 10(MB/s) for all virtual CPUs successfully
[root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit
<vcpu id='0' enabled='yes' hotpluggable='no' order='1'
dirty_limit='10'/>
<vcpu id='1' enabled='yes' hotpluggable='no' order='2'
dirty_limit='10'/>
<vcpu id='2' enabled='yes' hotpluggable='no' order='3'
dirty_limit='10'/>
<vcpu id='3' enabled='no' hotpluggable='yes' dirty_limit='10'/>
<vcpu id='4' enabled='no' hotpluggable='yes' dirty_limit='10'/>
......
Query the dirty limit info dynamically:
[root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit
Domain: 'c81_node1'
dirtylimit.vcpu.0.limit=10
dirtylimit.vcpu.0.current=0
dirtylimit.vcpu.1.limit=10
dirtylimit.vcpu.1.current=0
dirtylimit.vcpu.2.limit=10
dirtylimit.vcpu.2.current=0
dirtylimit.vcpu.3.limit=10
dirtylimit.vcpu.3.current=0
dirtylimit.vcpu.4.limit=10
dirtylimit.vcpu.4.current=0
......
To cancel the upper limit, use:
[root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 \
--rate 0 --live
Cancel dirty page rate limit for all virtual CPUs successfully
[root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit
[root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit
Domain: 'c81_node1'
The dirty limit uses the QEMU dirty-limit feature introduced since
7.1.0, this feature allows CPU to be throttled as needed to keep
their dirty page rate within the limit. It could, in some scenes, be
used to provide quality-of-service in the aspect of the memory
workload for virtual CPUs and QEMU itself use the feature to
implement the dirty-limit throttle algorithm and apply it on the
live migration, which improve responsiveness of large guests
during live migration and can result in more stable read
performance. The other application scenarios remain
unexplored, before that, Libvirt could provide the basic API.
Please review, thanks
Yong
Hyman Huang(黄勇) (10):
qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability
conf: Introduce XML for dirty limit configuration
libvirt: Add virDomainSetVcpuDirtyLimit API
qemu_driver: Implement qemuDomainSetVcpuDirtyLimit
domain_validate: Export virDomainDefHasDirtyLimitStartupVcpus symbol
qemu_process: Setup dirty limit after launching VM
virsh: Introduce limit-dirty-page-rate api
qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit
qemu_driver: Extend dirtlimit statistics for domGetStats
virsh: Introduce command 'virsh domstats --dirtylimit'
docs/formatdomain.rst | 7 +-
docs/manpages/virsh.rst | 33 +++-
include/libvirt/libvirt-domain.h | 5 +
src/conf/domain_conf.c | 26 +++
src/conf/domain_conf.h | 8 +
src/conf/domain_validate.c | 33 ++++
src/conf/domain_validate.h | 2 +
src/conf/schemas/domaincommon.rng | 5 +
src/driver-hypervisor.h | 7 +
src/libvirt-domain.c | 68 +++++++
src/libvirt_private.syms | 1 +
src/libvirt_public.syms | 5 +
src/qemu/qemu_capabilities.c | 2 +
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_driver.c | 181 ++++++++++++++++++
src/qemu/qemu_monitor.c | 25 +++
src/qemu/qemu_monitor.h | 22 +++
src/qemu/qemu_monitor_json.c | 107 +++++++++++
src/qemu/qemu_monitor_json.h | 9 +
src/qemu/qemu_process.c | 44 +++++
src/remote/remote_driver.c | 1 +
src/remote/remote_protocol.x | 17 +-
src/remote_protocol-structs | 7 +
.../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 +
.../caps_7.1.0_x86_64.xml | 1 +
tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 +
.../caps_7.2.0_x86_64+hvf.xml | 1 +
.../caps_7.2.0_x86_64.xml | 1 +
.../caps_8.0.0_riscv64.xml | 1 +
.../caps_8.0.0_x86_64.xml | 1 +
.../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 +
.../caps_8.1.0_x86_64.xml | 1 +
tools/virsh-domain-monitor.c | 7 +
tools/virsh-domain.c | 109 +++++++++++
34 files changed, 737 insertions(+), 4 deletions(-)
--
2.38.5
Ping. On Mon, Aug 7, 2023 at 11:56 PM ~hyman <hyman@git.sr.ht> wrote: > Hi, This is the latest version for the series, comparing with version > 1, there are some key modifications has made inspired and > suggested by Peter, see as follows: > 1. Introduce XML for dirty limit persistent configuration > 2. Merge the cancel API into the set API > 3. Extend the domstats/virDomainListGetStats API for dirty limit > information query > 4. Introduce the virDomainModificationImpact flags to control the > behavior of the API > 5. Enrich the comments and docs about the feature and API > > The patch set introduce the new API virDomainSetVcpuDirtyLimit to > allow upper Apps to set upper limits of dirty page rate for virtual > CPUs, > the corresponding virsh API as follows: > # limit-dirty-page-rate <domain> <rate> [--vcpu <number>] \ > [--config] [--live] [--current] > > We put the dirty limit persistent info with the "vcpus" element in > domain XML and > extend dirtylimit statistics for domGetStats: > <domain> > ... > <vcpu current='2'>3</vcpu> > <vcpus> > <vcpu id='0' hotpluggable='no' dirty_limit='10' order='1'.../> > <vcpu id='1' hotpluggable='yes' dirty_limit='10' order='2'.../> > </vcpus> > ... > > If --vcpu option is not passed in the virsh command, set all virtual > CPUs; > if rate is set to zero, cancel the upper limit. > > Examples: > To set the dirty page rate upper limit 10 MB/s for all virtual CPUs in > c81_node1, use: > [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 --rate 10 > --live > Set dirty page rate limit 10(MB/s) for all virtual CPUs successfully > > [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit > <vcpu id='0' enabled='yes' hotpluggable='no' order='1' > dirty_limit='10'/> > <vcpu id='1' enabled='yes' hotpluggable='no' order='2' > dirty_limit='10'/> > <vcpu id='2' enabled='yes' hotpluggable='no' order='3' > dirty_limit='10'/> > <vcpu id='3' enabled='no' hotpluggable='yes' dirty_limit='10'/> > <vcpu id='4' enabled='no' hotpluggable='yes' dirty_limit='10'/> > ...... > > Query the dirty limit info dynamically: > [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit > Domain: 'c81_node1' > dirtylimit.vcpu.0.limit=10 > dirtylimit.vcpu.0.current=0 > dirtylimit.vcpu.1.limit=10 > dirtylimit.vcpu.1.current=0 > dirtylimit.vcpu.2.limit=10 > dirtylimit.vcpu.2.current=0 > dirtylimit.vcpu.3.limit=10 > dirtylimit.vcpu.3.current=0 > dirtylimit.vcpu.4.limit=10 > dirtylimit.vcpu.4.current=0 > ...... > To cancel the upper limit, use: > [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 \ > --rate 0 --live > Cancel dirty page rate limit for all virtual CPUs successfully > > [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit > [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit > Domain: 'c81_node1' > > The dirty limit uses the QEMU dirty-limit feature introduced since > 7.1.0, this feature allows CPU to be throttled as needed to keep > their dirty page rate within the limit. It could, in some scenes, be > used to provide quality-of-service in the aspect of the memory > workload for virtual CPUs and QEMU itself use the feature to > implement the dirty-limit throttle algorithm and apply it on the > live migration, which improve responsiveness of large guests > during live migration and can result in more stable read > performance. The other application scenarios remain > unexplored, before that, Libvirt could provide the basic API. > > Please review, thanks > > Yong > > Hyman Huang(黄勇) (10): > qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability > conf: Introduce XML for dirty limit configuration > libvirt: Add virDomainSetVcpuDirtyLimit API > qemu_driver: Implement qemuDomainSetVcpuDirtyLimit > domain_validate: Export virDomainDefHasDirtyLimitStartupVcpus symbol > qemu_process: Setup dirty limit after launching VM > virsh: Introduce limit-dirty-page-rate api > qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit > qemu_driver: Extend dirtlimit statistics for domGetStats > virsh: Introduce command 'virsh domstats --dirtylimit' > > docs/formatdomain.rst | 7 +- > docs/manpages/virsh.rst | 33 +++- > include/libvirt/libvirt-domain.h | 5 + > src/conf/domain_conf.c | 26 +++ > src/conf/domain_conf.h | 8 + > src/conf/domain_validate.c | 33 ++++ > src/conf/domain_validate.h | 2 + > src/conf/schemas/domaincommon.rng | 5 + > src/driver-hypervisor.h | 7 + > src/libvirt-domain.c | 68 +++++++ > src/libvirt_private.syms | 1 + > src/libvirt_public.syms | 5 + > src/qemu/qemu_capabilities.c | 2 + > src/qemu/qemu_capabilities.h | 1 + > src/qemu/qemu_driver.c | 181 ++++++++++++++++++ > src/qemu/qemu_monitor.c | 25 +++ > src/qemu/qemu_monitor.h | 22 +++ > src/qemu/qemu_monitor_json.c | 107 +++++++++++ > src/qemu/qemu_monitor_json.h | 9 + > src/qemu/qemu_process.c | 44 +++++ > src/remote/remote_driver.c | 1 + > src/remote/remote_protocol.x | 17 +- > src/remote_protocol-structs | 7 + > .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 + > .../caps_7.1.0_x86_64.xml | 1 + > tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 + > .../caps_7.2.0_x86_64+hvf.xml | 1 + > .../caps_7.2.0_x86_64.xml | 1 + > .../caps_8.0.0_riscv64.xml | 1 + > .../caps_8.0.0_x86_64.xml | 1 + > .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 + > .../caps_8.1.0_x86_64.xml | 1 + > tools/virsh-domain-monitor.c | 7 + > tools/virsh-domain.c | 109 +++++++++++ > 34 files changed, 737 insertions(+), 4 deletions(-) > > -- > 2.38.5 > -- Best regards
Ping1 On Tue, Aug 15, 2023 at 9:48 AM Yong Huang <yong.huang@smartx.com> wrote: > Ping. > > On Mon, Aug 7, 2023 at 11:56 PM ~hyman <hyman@git.sr.ht> wrote: > >> Hi, This is the latest version for the series, comparing with version >> 1, there are some key modifications has made inspired and >> suggested by Peter, see as follows: >> 1. Introduce XML for dirty limit persistent configuration >> 2. Merge the cancel API into the set API >> 3. Extend the domstats/virDomainListGetStats API for dirty limit >> information query >> 4. Introduce the virDomainModificationImpact flags to control the >> behavior of the API >> 5. Enrich the comments and docs about the feature and API >> >> The patch set introduce the new API virDomainSetVcpuDirtyLimit to >> allow upper Apps to set upper limits of dirty page rate for virtual >> CPUs, >> the corresponding virsh API as follows: >> # limit-dirty-page-rate <domain> <rate> [--vcpu <number>] \ >> [--config] [--live] [--current] >> >> We put the dirty limit persistent info with the "vcpus" element in >> domain XML and >> extend dirtylimit statistics for domGetStats: >> <domain> >> ... >> <vcpu current='2'>3</vcpu> >> <vcpus> >> <vcpu id='0' hotpluggable='no' dirty_limit='10' order='1'.../> >> <vcpu id='1' hotpluggable='yes' dirty_limit='10' order='2'.../> >> </vcpus> >> ... >> >> If --vcpu option is not passed in the virsh command, set all virtual >> CPUs; >> if rate is set to zero, cancel the upper limit. >> >> Examples: >> To set the dirty page rate upper limit 10 MB/s for all virtual CPUs in >> c81_node1, use: >> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 --rate 10 >> --live >> Set dirty page rate limit 10(MB/s) for all virtual CPUs successfully >> >> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit >> <vcpu id='0' enabled='yes' hotpluggable='no' order='1' >> dirty_limit='10'/> >> <vcpu id='1' enabled='yes' hotpluggable='no' order='2' >> dirty_limit='10'/> >> <vcpu id='2' enabled='yes' hotpluggable='no' order='3' >> dirty_limit='10'/> >> <vcpu id='3' enabled='no' hotpluggable='yes' dirty_limit='10'/> >> <vcpu id='4' enabled='no' hotpluggable='yes' dirty_limit='10'/> >> ...... >> >> Query the dirty limit info dynamically: >> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit >> Domain: 'c81_node1' >> dirtylimit.vcpu.0.limit=10 >> dirtylimit.vcpu.0.current=0 >> dirtylimit.vcpu.1.limit=10 >> dirtylimit.vcpu.1.current=0 >> dirtylimit.vcpu.2.limit=10 >> dirtylimit.vcpu.2.current=0 >> dirtylimit.vcpu.3.limit=10 >> dirtylimit.vcpu.3.current=0 >> dirtylimit.vcpu.4.limit=10 >> dirtylimit.vcpu.4.current=0 >> ...... >> To cancel the upper limit, use: >> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 \ >> --rate 0 --live >> Cancel dirty page rate limit for all virtual CPUs successfully >> >> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit >> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit >> Domain: 'c81_node1' >> >> The dirty limit uses the QEMU dirty-limit feature introduced since >> 7.1.0, this feature allows CPU to be throttled as needed to keep >> their dirty page rate within the limit. It could, in some scenes, be >> used to provide quality-of-service in the aspect of the memory >> workload for virtual CPUs and QEMU itself use the feature to >> implement the dirty-limit throttle algorithm and apply it on the >> live migration, which improve responsiveness of large guests >> during live migration and can result in more stable read >> performance. The other application scenarios remain >> unexplored, before that, Libvirt could provide the basic API. >> >> Please review, thanks >> >> Yong >> >> Hyman Huang(黄勇) (10): >> qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability >> conf: Introduce XML for dirty limit configuration >> libvirt: Add virDomainSetVcpuDirtyLimit API >> qemu_driver: Implement qemuDomainSetVcpuDirtyLimit >> domain_validate: Export virDomainDefHasDirtyLimitStartupVcpus symbol >> qemu_process: Setup dirty limit after launching VM >> virsh: Introduce limit-dirty-page-rate api >> qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit >> qemu_driver: Extend dirtlimit statistics for domGetStats >> virsh: Introduce command 'virsh domstats --dirtylimit' >> >> docs/formatdomain.rst | 7 +- >> docs/manpages/virsh.rst | 33 +++- >> include/libvirt/libvirt-domain.h | 5 + >> src/conf/domain_conf.c | 26 +++ >> src/conf/domain_conf.h | 8 + >> src/conf/domain_validate.c | 33 ++++ >> src/conf/domain_validate.h | 2 + >> src/conf/schemas/domaincommon.rng | 5 + >> src/driver-hypervisor.h | 7 + >> src/libvirt-domain.c | 68 +++++++ >> src/libvirt_private.syms | 1 + >> src/libvirt_public.syms | 5 + >> src/qemu/qemu_capabilities.c | 2 + >> src/qemu/qemu_capabilities.h | 1 + >> src/qemu/qemu_driver.c | 181 ++++++++++++++++++ >> src/qemu/qemu_monitor.c | 25 +++ >> src/qemu/qemu_monitor.h | 22 +++ >> src/qemu/qemu_monitor_json.c | 107 +++++++++++ >> src/qemu/qemu_monitor_json.h | 9 + >> src/qemu/qemu_process.c | 44 +++++ >> src/remote/remote_driver.c | 1 + >> src/remote/remote_protocol.x | 17 +- >> src/remote_protocol-structs | 7 + >> .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 + >> .../caps_7.1.0_x86_64.xml | 1 + >> tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 + >> .../caps_7.2.0_x86_64+hvf.xml | 1 + >> .../caps_7.2.0_x86_64.xml | 1 + >> .../caps_8.0.0_riscv64.xml | 1 + >> .../caps_8.0.0_x86_64.xml | 1 + >> .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 + >> .../caps_8.1.0_x86_64.xml | 1 + >> tools/virsh-domain-monitor.c | 7 + >> tools/virsh-domain.c | 109 +++++++++++ >> 34 files changed, 737 insertions(+), 4 deletions(-) >> >> -- >> 2.38.5 >> > > > -- > Best regards > -- Best regards
Ping2, I'm hoping for comments about the series. Thanks, Yong On Sun, Aug 27, 2023 at 11:11 AM Yong Huang <yong.huang@smartx.com> wrote: > Ping1 > > On Tue, Aug 15, 2023 at 9:48 AM Yong Huang <yong.huang@smartx.com> wrote: > >> Ping. >> >> On Mon, Aug 7, 2023 at 11:56 PM ~hyman <hyman@git.sr.ht> wrote: >> >>> Hi, This is the latest version for the series, comparing with version >>> 1, there are some key modifications has made inspired and >>> suggested by Peter, see as follows: >>> 1. Introduce XML for dirty limit persistent configuration >>> 2. Merge the cancel API into the set API >>> 3. Extend the domstats/virDomainListGetStats API for dirty limit >>> information query >>> 4. Introduce the virDomainModificationImpact flags to control the >>> behavior of the API >>> 5. Enrich the comments and docs about the feature and API >>> >>> The patch set introduce the new API virDomainSetVcpuDirtyLimit to >>> allow upper Apps to set upper limits of dirty page rate for virtual >>> CPUs, >>> the corresponding virsh API as follows: >>> # limit-dirty-page-rate <domain> <rate> [--vcpu <number>] \ >>> [--config] [--live] [--current] >>> >>> We put the dirty limit persistent info with the "vcpus" element in >>> domain XML and >>> extend dirtylimit statistics for domGetStats: >>> <domain> >>> ... >>> <vcpu current='2'>3</vcpu> >>> <vcpus> >>> <vcpu id='0' hotpluggable='no' dirty_limit='10' order='1'.../> >>> <vcpu id='1' hotpluggable='yes' dirty_limit='10' order='2'.../> >>> </vcpus> >>> ... >>> >>> If --vcpu option is not passed in the virsh command, set all virtual >>> CPUs; >>> if rate is set to zero, cancel the upper limit. >>> >>> Examples: >>> To set the dirty page rate upper limit 10 MB/s for all virtual CPUs in >>> c81_node1, use: >>> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 --rate 10 >>> --live >>> Set dirty page rate limit 10(MB/s) for all virtual CPUs successfully >>> >>> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit >>> <vcpu id='0' enabled='yes' hotpluggable='no' order='1' >>> dirty_limit='10'/> >>> <vcpu id='1' enabled='yes' hotpluggable='no' order='2' >>> dirty_limit='10'/> >>> <vcpu id='2' enabled='yes' hotpluggable='no' order='3' >>> dirty_limit='10'/> >>> <vcpu id='3' enabled='no' hotpluggable='yes' dirty_limit='10'/> >>> <vcpu id='4' enabled='no' hotpluggable='yes' dirty_limit='10'/> >>> ...... >>> >>> Query the dirty limit info dynamically: >>> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit >>> Domain: 'c81_node1' >>> dirtylimit.vcpu.0.limit=10 >>> dirtylimit.vcpu.0.current=0 >>> dirtylimit.vcpu.1.limit=10 >>> dirtylimit.vcpu.1.current=0 >>> dirtylimit.vcpu.2.limit=10 >>> dirtylimit.vcpu.2.current=0 >>> dirtylimit.vcpu.3.limit=10 >>> dirtylimit.vcpu.3.current=0 >>> dirtylimit.vcpu.4.limit=10 >>> dirtylimit.vcpu.4.current=0 >>> ...... >>> To cancel the upper limit, use: >>> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 \ >>> --rate 0 --live >>> Cancel dirty page rate limit for all virtual CPUs successfully >>> >>> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit >>> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit >>> Domain: 'c81_node1' >>> >>> The dirty limit uses the QEMU dirty-limit feature introduced since >>> 7.1.0, this feature allows CPU to be throttled as needed to keep >>> their dirty page rate within the limit. It could, in some scenes, be >>> used to provide quality-of-service in the aspect of the memory >>> workload for virtual CPUs and QEMU itself use the feature to >>> implement the dirty-limit throttle algorithm and apply it on the >>> live migration, which improve responsiveness of large guests >>> during live migration and can result in more stable read >>> performance. The other application scenarios remain >>> unexplored, before that, Libvirt could provide the basic API. >>> >>> Please review, thanks >>> >>> Yong >>> >>> Hyman Huang(黄勇) (10): >>> qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability >>> conf: Introduce XML for dirty limit configuration >>> libvirt: Add virDomainSetVcpuDirtyLimit API >>> qemu_driver: Implement qemuDomainSetVcpuDirtyLimit >>> domain_validate: Export virDomainDefHasDirtyLimitStartupVcpus symbol >>> qemu_process: Setup dirty limit after launching VM >>> virsh: Introduce limit-dirty-page-rate api >>> qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit >>> qemu_driver: Extend dirtlimit statistics for domGetStats >>> virsh: Introduce command 'virsh domstats --dirtylimit' >>> >>> docs/formatdomain.rst | 7 +- >>> docs/manpages/virsh.rst | 33 +++- >>> include/libvirt/libvirt-domain.h | 5 + >>> src/conf/domain_conf.c | 26 +++ >>> src/conf/domain_conf.h | 8 + >>> src/conf/domain_validate.c | 33 ++++ >>> src/conf/domain_validate.h | 2 + >>> src/conf/schemas/domaincommon.rng | 5 + >>> src/driver-hypervisor.h | 7 + >>> src/libvirt-domain.c | 68 +++++++ >>> src/libvirt_private.syms | 1 + >>> src/libvirt_public.syms | 5 + >>> src/qemu/qemu_capabilities.c | 2 + >>> src/qemu/qemu_capabilities.h | 1 + >>> src/qemu/qemu_driver.c | 181 ++++++++++++++++++ >>> src/qemu/qemu_monitor.c | 25 +++ >>> src/qemu/qemu_monitor.h | 22 +++ >>> src/qemu/qemu_monitor_json.c | 107 +++++++++++ >>> src/qemu/qemu_monitor_json.h | 9 + >>> src/qemu/qemu_process.c | 44 +++++ >>> src/remote/remote_driver.c | 1 + >>> src/remote/remote_protocol.x | 17 +- >>> src/remote_protocol-structs | 7 + >>> .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 + >>> .../caps_7.1.0_x86_64.xml | 1 + >>> tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 + >>> .../caps_7.2.0_x86_64+hvf.xml | 1 + >>> .../caps_7.2.0_x86_64.xml | 1 + >>> .../caps_8.0.0_riscv64.xml | 1 + >>> .../caps_8.0.0_x86_64.xml | 1 + >>> .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 + >>> .../caps_8.1.0_x86_64.xml | 1 + >>> tools/virsh-domain-monitor.c | 7 + >>> tools/virsh-domain.c | 109 +++++++++++ >>> 34 files changed, 737 insertions(+), 4 deletions(-) >>> >>> -- >>> 2.38.5 >>> >> >> >> -- >> Best regards >> > > > -- > Best regards > -- Best regards
Sorry for not looking into this earlier, but it's been quite a while and I, personally, received only patches 2, 6, 5, 9, and 10 from this series. I, however, see the rest in the archive, so the issue is probably somewhere on my part. Would you mind resending the second version again, ideally rebased? Thanks, Martin On Mon, Sep 04, 2023 at 09:32:08PM +0800, Yong Huang wrote: >Ping2, I'm hoping for comments about the series. > >Thanks, >Yong > >On Sun, Aug 27, 2023 at 11:11 AM Yong Huang <yong.huang@smartx.com> wrote: > >> Ping1 >> >> On Tue, Aug 15, 2023 at 9:48 AM Yong Huang <yong.huang@smartx.com> wrote: >> >>> Ping. >>> >>> On Mon, Aug 7, 2023 at 11:56 PM ~hyman <hyman@git.sr.ht> wrote: >>> >>>> Hi, This is the latest version for the series, comparing with version >>>> 1, there are some key modifications has made inspired and >>>> suggested by Peter, see as follows: >>>> 1. Introduce XML for dirty limit persistent configuration >>>> 2. Merge the cancel API into the set API >>>> 3. Extend the domstats/virDomainListGetStats API for dirty limit >>>> information query >>>> 4. Introduce the virDomainModificationImpact flags to control the >>>> behavior of the API >>>> 5. Enrich the comments and docs about the feature and API >>>> >>>> The patch set introduce the new API virDomainSetVcpuDirtyLimit to >>>> allow upper Apps to set upper limits of dirty page rate for virtual >>>> CPUs, >>>> the corresponding virsh API as follows: >>>> # limit-dirty-page-rate <domain> <rate> [--vcpu <number>] \ >>>> [--config] [--live] [--current] >>>> >>>> We put the dirty limit persistent info with the "vcpus" element in >>>> domain XML and >>>> extend dirtylimit statistics for domGetStats: >>>> <domain> >>>> ... >>>> <vcpu current='2'>3</vcpu> >>>> <vcpus> >>>> <vcpu id='0' hotpluggable='no' dirty_limit='10' order='1'.../> >>>> <vcpu id='1' hotpluggable='yes' dirty_limit='10' order='2'.../> >>>> </vcpus> >>>> ... >>>> >>>> If --vcpu option is not passed in the virsh command, set all virtual >>>> CPUs; >>>> if rate is set to zero, cancel the upper limit. >>>> >>>> Examples: >>>> To set the dirty page rate upper limit 10 MB/s for all virtual CPUs in >>>> c81_node1, use: >>>> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 --rate 10 >>>> --live >>>> Set dirty page rate limit 10(MB/s) for all virtual CPUs successfully >>>> >>>> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit >>>> <vcpu id='0' enabled='yes' hotpluggable='no' order='1' >>>> dirty_limit='10'/> >>>> <vcpu id='1' enabled='yes' hotpluggable='no' order='2' >>>> dirty_limit='10'/> >>>> <vcpu id='2' enabled='yes' hotpluggable='no' order='3' >>>> dirty_limit='10'/> >>>> <vcpu id='3' enabled='no' hotpluggable='yes' dirty_limit='10'/> >>>> <vcpu id='4' enabled='no' hotpluggable='yes' dirty_limit='10'/> >>>> ...... >>>> >>>> Query the dirty limit info dynamically: >>>> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit >>>> Domain: 'c81_node1' >>>> dirtylimit.vcpu.0.limit=10 >>>> dirtylimit.vcpu.0.current=0 >>>> dirtylimit.vcpu.1.limit=10 >>>> dirtylimit.vcpu.1.current=0 >>>> dirtylimit.vcpu.2.limit=10 >>>> dirtylimit.vcpu.2.current=0 >>>> dirtylimit.vcpu.3.limit=10 >>>> dirtylimit.vcpu.3.current=0 >>>> dirtylimit.vcpu.4.limit=10 >>>> dirtylimit.vcpu.4.current=0 >>>> ...... >>>> To cancel the upper limit, use: >>>> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 \ >>>> --rate 0 --live >>>> Cancel dirty page rate limit for all virtual CPUs successfully >>>> >>>> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit >>>> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit >>>> Domain: 'c81_node1' >>>> >>>> The dirty limit uses the QEMU dirty-limit feature introduced since >>>> 7.1.0, this feature allows CPU to be throttled as needed to keep >>>> their dirty page rate within the limit. It could, in some scenes, be >>>> used to provide quality-of-service in the aspect of the memory >>>> workload for virtual CPUs and QEMU itself use the feature to >>>> implement the dirty-limit throttle algorithm and apply it on the >>>> live migration, which improve responsiveness of large guests >>>> during live migration and can result in more stable read >>>> performance. The other application scenarios remain >>>> unexplored, before that, Libvirt could provide the basic API. >>>> >>>> Please review, thanks >>>> >>>> Yong >>>> >>>> Hyman Huang(黄勇) (10): >>>> qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability >>>> conf: Introduce XML for dirty limit configuration >>>> libvirt: Add virDomainSetVcpuDirtyLimit API >>>> qemu_driver: Implement qemuDomainSetVcpuDirtyLimit >>>> domain_validate: Export virDomainDefHasDirtyLimitStartupVcpus symbol >>>> qemu_process: Setup dirty limit after launching VM >>>> virsh: Introduce limit-dirty-page-rate api >>>> qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit >>>> qemu_driver: Extend dirtlimit statistics for domGetStats >>>> virsh: Introduce command 'virsh domstats --dirtylimit' >>>> >>>> docs/formatdomain.rst | 7 +- >>>> docs/manpages/virsh.rst | 33 +++- >>>> include/libvirt/libvirt-domain.h | 5 + >>>> src/conf/domain_conf.c | 26 +++ >>>> src/conf/domain_conf.h | 8 + >>>> src/conf/domain_validate.c | 33 ++++ >>>> src/conf/domain_validate.h | 2 + >>>> src/conf/schemas/domaincommon.rng | 5 + >>>> src/driver-hypervisor.h | 7 + >>>> src/libvirt-domain.c | 68 +++++++ >>>> src/libvirt_private.syms | 1 + >>>> src/libvirt_public.syms | 5 + >>>> src/qemu/qemu_capabilities.c | 2 + >>>> src/qemu/qemu_capabilities.h | 1 + >>>> src/qemu/qemu_driver.c | 181 ++++++++++++++++++ >>>> src/qemu/qemu_monitor.c | 25 +++ >>>> src/qemu/qemu_monitor.h | 22 +++ >>>> src/qemu/qemu_monitor_json.c | 107 +++++++++++ >>>> src/qemu/qemu_monitor_json.h | 9 + >>>> src/qemu/qemu_process.c | 44 +++++ >>>> src/remote/remote_driver.c | 1 + >>>> src/remote/remote_protocol.x | 17 +- >>>> src/remote_protocol-structs | 7 + >>>> .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 + >>>> .../caps_7.1.0_x86_64.xml | 1 + >>>> tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 + >>>> .../caps_7.2.0_x86_64+hvf.xml | 1 + >>>> .../caps_7.2.0_x86_64.xml | 1 + >>>> .../caps_8.0.0_riscv64.xml | 1 + >>>> .../caps_8.0.0_x86_64.xml | 1 + >>>> .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 + >>>> .../caps_8.1.0_x86_64.xml | 1 + >>>> tools/virsh-domain-monitor.c | 7 + >>>> tools/virsh-domain.c | 109 +++++++++++ >>>> 34 files changed, 737 insertions(+), 4 deletions(-) >>>> >>>> -- >>>> 2.38.5 >>>> >>> >>> >>> -- >>> Best regards >>> >> >> >> -- >> Best regards >> > > >-- >Best regards
On Tue, Sep 5, 2023 at 6:22 PM Martin Kletzander <mkletzan@redhat.com> wrote: > Sorry for not looking into this earlier, but it's been quite a while and > I, personally, received only patches 2, 6, 5, 9, and 10 from this > series. I, however, see the rest in the archive, so the issue is > probably somewhere on my part. > > Would you mind resending the second version again, ideally rebased? > Sure yes, I'll rebase the master and resend in the near future Thanks, Yong > > Thanks, > Martin > > On Mon, Sep 04, 2023 at 09:32:08PM +0800, Yong Huang wrote: > >Ping2, I'm hoping for comments about the series. > > > >Thanks, > >Yong > > > >On Sun, Aug 27, 2023 at 11:11 AM Yong Huang <yong.huang@smartx.com> > wrote: > > > >> Ping1 > >> > >> On Tue, Aug 15, 2023 at 9:48 AM Yong Huang <yong.huang@smartx.com> > wrote: > >> > >>> Ping. > >>> > >>> On Mon, Aug 7, 2023 at 11:56 PM ~hyman <hyman@git.sr.ht> wrote: > >>> > >>>> Hi, This is the latest version for the series, comparing with version > >>>> 1, there are some key modifications has made inspired and > >>>> suggested by Peter, see as follows: > >>>> 1. Introduce XML for dirty limit persistent configuration > >>>> 2. Merge the cancel API into the set API > >>>> 3. Extend the domstats/virDomainListGetStats API for dirty limit > >>>> information query > >>>> 4. Introduce the virDomainModificationImpact flags to control the > >>>> behavior of the API > >>>> 5. Enrich the comments and docs about the feature and API > >>>> > >>>> The patch set introduce the new API virDomainSetVcpuDirtyLimit to > >>>> allow upper Apps to set upper limits of dirty page rate for virtual > >>>> CPUs, > >>>> the corresponding virsh API as follows: > >>>> # limit-dirty-page-rate <domain> <rate> [--vcpu <number>] \ > >>>> [--config] [--live] [--current] > >>>> > >>>> We put the dirty limit persistent info with the "vcpus" element in > >>>> domain XML and > >>>> extend dirtylimit statistics for domGetStats: > >>>> <domain> > >>>> ... > >>>> <vcpu current='2'>3</vcpu> > >>>> <vcpus> > >>>> <vcpu id='0' hotpluggable='no' dirty_limit='10' order='1'.../> > >>>> <vcpu id='1' hotpluggable='yes' dirty_limit='10' > order='2'.../> > >>>> </vcpus> > >>>> ... > >>>> > >>>> If --vcpu option is not passed in the virsh command, set all virtual > >>>> CPUs; > >>>> if rate is set to zero, cancel the upper limit. > >>>> > >>>> Examples: > >>>> To set the dirty page rate upper limit 10 MB/s for all virtual CPUs in > >>>> c81_node1, use: > >>>> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 --rate > 10 > >>>> --live > >>>> Set dirty page rate limit 10(MB/s) for all virtual CPUs successfully > >>>> > >>>> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit > >>>> <vcpu id='0' enabled='yes' hotpluggable='no' order='1' > >>>> dirty_limit='10'/> > >>>> <vcpu id='1' enabled='yes' hotpluggable='no' order='2' > >>>> dirty_limit='10'/> > >>>> <vcpu id='2' enabled='yes' hotpluggable='no' order='3' > >>>> dirty_limit='10'/> > >>>> <vcpu id='3' enabled='no' hotpluggable='yes' dirty_limit='10'/> > >>>> <vcpu id='4' enabled='no' hotpluggable='yes' dirty_limit='10'/> > >>>> ...... > >>>> > >>>> Query the dirty limit info dynamically: > >>>> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit > >>>> Domain: 'c81_node1' > >>>> dirtylimit.vcpu.0.limit=10 > >>>> dirtylimit.vcpu.0.current=0 > >>>> dirtylimit.vcpu.1.limit=10 > >>>> dirtylimit.vcpu.1.current=0 > >>>> dirtylimit.vcpu.2.limit=10 > >>>> dirtylimit.vcpu.2.current=0 > >>>> dirtylimit.vcpu.3.limit=10 > >>>> dirtylimit.vcpu.3.current=0 > >>>> dirtylimit.vcpu.4.limit=10 > >>>> dirtylimit.vcpu.4.current=0 > >>>> ...... > >>>> To cancel the upper limit, use: > >>>> [root@srv2 my_libvirt]# virsh limit-dirty-page-rate c81_node1 \ > >>>> --rate 0 --live > >>>> Cancel dirty page rate limit for all virtual CPUs successfully > >>>> > >>>> [root@srv2 my_libvirt]# virsh dumpxml c81_node1 | grep dirty_limit > >>>> [root@srv2 my_libvirt]# virsh domstats c81_node1 --dirtylimit > >>>> Domain: 'c81_node1' > >>>> > >>>> The dirty limit uses the QEMU dirty-limit feature introduced since > >>>> 7.1.0, this feature allows CPU to be throttled as needed to keep > >>>> their dirty page rate within the limit. It could, in some scenes, be > >>>> used to provide quality-of-service in the aspect of the memory > >>>> workload for virtual CPUs and QEMU itself use the feature to > >>>> implement the dirty-limit throttle algorithm and apply it on the > >>>> live migration, which improve responsiveness of large guests > >>>> during live migration and can result in more stable read > >>>> performance. The other application scenarios remain > >>>> unexplored, before that, Libvirt could provide the basic API. > >>>> > >>>> Please review, thanks > >>>> > >>>> Yong > >>>> > >>>> Hyman Huang(黄勇) (10): > >>>> qemu_capabilities: Introduce QEMU_CAPS_VCPU_DIRTY_LIMIT capability > >>>> conf: Introduce XML for dirty limit configuration > >>>> libvirt: Add virDomainSetVcpuDirtyLimit API > >>>> qemu_driver: Implement qemuDomainSetVcpuDirtyLimit > >>>> domain_validate: Export virDomainDefHasDirtyLimitStartupVcpus symbol > >>>> qemu_process: Setup dirty limit after launching VM > >>>> virsh: Introduce limit-dirty-page-rate api > >>>> qemu_monitor: Implement qemuMonitorQueryVcpuDirtyLimit > >>>> qemu_driver: Extend dirtlimit statistics for domGetStats > >>>> virsh: Introduce command 'virsh domstats --dirtylimit' > >>>> > >>>> docs/formatdomain.rst | 7 +- > >>>> docs/manpages/virsh.rst | 33 +++- > >>>> include/libvirt/libvirt-domain.h | 5 + > >>>> src/conf/domain_conf.c | 26 +++ > >>>> src/conf/domain_conf.h | 8 + > >>>> src/conf/domain_validate.c | 33 ++++ > >>>> src/conf/domain_validate.h | 2 + > >>>> src/conf/schemas/domaincommon.rng | 5 + > >>>> src/driver-hypervisor.h | 7 + > >>>> src/libvirt-domain.c | 68 +++++++ > >>>> src/libvirt_private.syms | 1 + > >>>> src/libvirt_public.syms | 5 + > >>>> src/qemu/qemu_capabilities.c | 2 + > >>>> src/qemu/qemu_capabilities.h | 1 + > >>>> src/qemu/qemu_driver.c | 181 > ++++++++++++++++++ > >>>> src/qemu/qemu_monitor.c | 25 +++ > >>>> src/qemu/qemu_monitor.h | 22 +++ > >>>> src/qemu/qemu_monitor_json.c | 107 +++++++++++ > >>>> src/qemu/qemu_monitor_json.h | 9 + > >>>> src/qemu/qemu_process.c | 44 +++++ > >>>> src/remote/remote_driver.c | 1 + > >>>> src/remote/remote_protocol.x | 17 +- > >>>> src/remote_protocol-structs | 7 + > >>>> .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 1 + > >>>> .../caps_7.1.0_x86_64.xml | 1 + > >>>> tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 1 + > >>>> .../caps_7.2.0_x86_64+hvf.xml | 1 + > >>>> .../caps_7.2.0_x86_64.xml | 1 + > >>>> .../caps_8.0.0_riscv64.xml | 1 + > >>>> .../caps_8.0.0_x86_64.xml | 1 + > >>>> .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 1 + > >>>> .../caps_8.1.0_x86_64.xml | 1 + > >>>> tools/virsh-domain-monitor.c | 7 + > >>>> tools/virsh-domain.c | 109 +++++++++++ > >>>> 34 files changed, 737 insertions(+), 4 deletions(-) > >>>> > >>>> -- > >>>> 2.38.5 > >>>> > >>> > >>> > >>> -- > >>> Best regards > >>> > >> > >> > >> -- > >> Best regards > >> > > > > > >-- > >Best regards > -- Best regards
© 2016 - 2024 Red Hat, Inc.