[libvirt] [v6 00/10] Support cache tune in libvirt

Eli Qiao posted 10 patches 7 years, 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/1487137313-87728-1-git-send-email-liyong.qiao@intel.com
There is a newer version of this series
docs/schemas/domaincommon.rng |   46 ++
include/libvirt/virterror.h   |    1 +
po/POTFILES.in                |    1 +
src/Makefile.am               |    1 +
src/conf/capabilities.c       |   56 ++
src/conf/capabilities.h       |   23 +
src/conf/domain_conf.c        |  182 +++++++
src/conf/domain_conf.h        |   19 +
src/libvirt_private.syms      |   11 +
src/nodeinfo.c                |   64 +++
src/nodeinfo.h                |    1 +
src/qemu/qemu_capabilities.c  |    8 +
src/qemu/qemu_driver.c        |    6 +
src/qemu/qemu_process.c       |   53 ++
src/util/virerror.c           |    1 +
src/util/virresctrl.c         | 1208 +++++++++++++++++++++++++++++++++++++++++
src/util/virresctrl.h         |   88 +++
17 files changed, 1769 insertions(+)
create mode 100644 src/util/virresctrl.c
create mode 100644 src/util/virresctrl.h
[libvirt] [v6 00/10] Support cache tune in libvirt
Posted by Eli Qiao 7 years, 1 month ago
Addressed comment from v6 -> v5

Marcelo:
       * Support other APPs to operate /sys/fs/resctrl at same time

         Libvirt will scan /sys/fs/resctrl again before doing cache allocation.
         patch 10 will address this.

Addressed comment from v4 -> v5:

Marcelo:
       * Several typos
       * Use flock instead of virFileLock

Addressed comment from v3 -> v4:

Daniel & Marcelo:
       * Added concurrence support

Addressed comment from v2 -> v3:

Daniel:
      * Fixed coding style, passed `make check` and `make syntax-check`

      * Variables renaming and move from header file to c file.

      * For locking/mutex support, no progress.

      There are some discussion from mailing list, but I can not find a better
      way to add locking support without performance impact.

      I'll explain the process and please help to advice what shoud we do.

      VM create:
      1) Get the cache left value on each bank of the host. This should be
         shared amount all VMs.
      2) Calculate the schemata on the bank based on all created resctrl
         domain's schemata
      3) Calculate the default schemata by scaning all domain's schemata.
      4) Flush default schemata to /sys/fs/resctrl/schemata

      VM destroy:
      1) Remove the resctrl domain of that VM
      2) Recalculate default schemata
      3) Flush default schemata to /sys/fs/resctrl/schemata

      The key point is that all VMs shares /sys/fs/resctrl/schemata, and
      when a VM create a resctrl domain, the schemata of that VM depends on
      the default schemata and all other exsited schematas. So a global
      mutex is reqired.

      Before calculate a schemata or update default schemata, libvirt
      should gain this global mutex.

      I will try to think more about how to support this gracefully in next
      patch set.

Marcelo:
      * Added vcpu support for cachetune, this will allow user to define which
        vcpu using which cache allocation bank.

        <cachetune id='0' host_id=0 size='3072' unit='KiB' vcpus='0,1'/>

        vcpus is a cpumap, the vcpu pids will be added to tasks file

      * Added cdp compatible, user can specify l3 cache even host enable cdp.
        See patch 8.
        On a cdp enabled host, specify l3code/l3data by

        <cachetune id='0' host_id='0' type='l3' size='3072' unit='KiB'/>

        This will create a schemata like:
            L3data:0=0xff00;...
            L3code:0=0xff00;...

      * Would you please help to test if the functions work.

Martin:
      * Xml test case, I have no time to work on this yet, would you please
        show me an example, would like to amend it later.

This series patches are for supportting CAT featues, which also
called cache tune in libvirt.

First to expose cache information which could be tuned in capabilites XML.
Then add new domain xml element support to add cacahe bank which will apply
on this libvirt domain.

This series patches add a util file `resctrl.c/h`, an interface to talk with
linux kernel's system fs.

There are still one TODO left:
    1. Expose a new public interface to get free cache information.
    2. Expose a new public interface to set cachetune lively.

Some discussion about this feature support can be found from:
https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html


Eli Qiao (10):
  Resctrl: Add some utils functions
  Resctrl: expose cache information to capabilities
  Resctrl: Add new xml element to support cache tune
  Resctrl: Add private interface to set cachebanks
  Qemu: Set cache banks
  Resctrl: enable l3code/l3data
  Resctrl: Make sure l3data/l3code are pairs
  Resctrl: Compatible mode for cdp enabled
  Resctrl: concurrence support
  Resctrl: Scan resctrl before doing cache allocation

 docs/schemas/domaincommon.rng |   46 ++
 include/libvirt/virterror.h   |    1 +
 po/POTFILES.in                |    1 +
 src/Makefile.am               |    1 +
 src/conf/capabilities.c       |   56 ++
 src/conf/capabilities.h       |   23 +
 src/conf/domain_conf.c        |  182 +++++++
 src/conf/domain_conf.h        |   19 +
 src/libvirt_private.syms      |   11 +
 src/nodeinfo.c                |   64 +++
 src/nodeinfo.h                |    1 +
 src/qemu/qemu_capabilities.c  |    8 +
 src/qemu/qemu_driver.c        |    6 +
 src/qemu/qemu_process.c       |   53 ++
 src/util/virerror.c           |    1 +
 src/util/virresctrl.c         | 1208 +++++++++++++++++++++++++++++++++++++++++
 src/util/virresctrl.h         |   88 +++
 17 files changed, 1769 insertions(+)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h

-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [v6 00/10] Support cache tune in libvirt
Posted by Martin Kletzander 7 years, 1 month ago
On Wed, Feb 15, 2017 at 01:41:43PM +0800, Eli Qiao wrote:
>Addressed comment from v6 -> v5
>
>Marcelo:
>       * Support other APPs to operate /sys/fs/resctrl at same time
>
>         Libvirt will scan /sys/fs/resctrl again before doing cache allocation.
>         patch 10 will address this.
>
>Addressed comment from v4 -> v5:
>
>Marcelo:
>       * Several typos
>       * Use flock instead of virFileLock
>
>Addressed comment from v3 -> v4:
>
>Daniel & Marcelo:
>       * Added concurrence support
>
>Addressed comment from v2 -> v3:
>
>Daniel:
>      * Fixed coding style, passed `make check` and `make syntax-check`
>
>      * Variables renaming and move from header file to c file.
>
>      * For locking/mutex support, no progress.
>
>      There are some discussion from mailing list, but I can not find a better
>      way to add locking support without performance impact.
>
>      I'll explain the process and please help to advice what shoud we do.
>
>      VM create:
>      1) Get the cache left value on each bank of the host. This should be
>         shared amount all VMs.
>      2) Calculate the schemata on the bank based on all created resctrl
>         domain's schemata
>      3) Calculate the default schemata by scaning all domain's schemata.
>      4) Flush default schemata to /sys/fs/resctrl/schemata
>
>      VM destroy:
>      1) Remove the resctrl domain of that VM
>      2) Recalculate default schemata
>      3) Flush default schemata to /sys/fs/resctrl/schemata
>
>      The key point is that all VMs shares /sys/fs/resctrl/schemata, and
>      when a VM create a resctrl domain, the schemata of that VM depends on
>      the default schemata and all other exsited schematas. So a global
>      mutex is reqired.
>
>      Before calculate a schemata or update default schemata, libvirt
>      should gain this global mutex.
>
>      I will try to think more about how to support this gracefully in next
>      patch set.
>
>Marcelo:
>      * Added vcpu support for cachetune, this will allow user to define which
>        vcpu using which cache allocation bank.
>
>        <cachetune id='0' host_id=0 size='3072' unit='KiB' vcpus='0,1'/>
>
>        vcpus is a cpumap, the vcpu pids will be added to tasks file
>
>      * Added cdp compatible, user can specify l3 cache even host enable cdp.
>        See patch 8.
>        On a cdp enabled host, specify l3code/l3data by
>
>        <cachetune id='0' host_id='0' type='l3' size='3072' unit='KiB'/>
>
>        This will create a schemata like:
>            L3data:0=0xff00;...
>            L3code:0=0xff00;...
>
>      * Would you please help to test if the functions work.
>
>Martin:
>      * Xml test case, I have no time to work on this yet, would you please
>        show me an example, would like to amend it later.
>

look at tests/qemuxml2xmltest, it is called 'qemu...', but it actually
just parses XML and then formats it back and checks the result (that
there is no information lost.  Input XMLs are in tests/qemuxml2argvdata/
and the desired outputs are in tests/qemuxml2xmloutdata/.  For XMLs that
should not change, the output XML is a symlink to the input one.  For
negative tests, there doesn't have to be any output XML (of course).

>This series patches are for supportting CAT featues, which also
>called cache tune in libvirt.
>
>First to expose cache information which could be tuned in capabilites XML.
>Then add new domain xml element support to add cacahe bank which will apply
>on this libvirt domain.
>
>This series patches add a util file `resctrl.c/h`, an interface to talk with
>linux kernel's system fs.
>
>There are still one TODO left:
>    1. Expose a new public interface to get free cache information.
>    2. Expose a new public interface to set cachetune lively.
>
>Some discussion about this feature support can be found from:
>https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html
>
>
>Eli Qiao (10):
>  Resctrl: Add some utils functions
>  Resctrl: expose cache information to capabilities
>  Resctrl: Add new xml element to support cache tune
>  Resctrl: Add private interface to set cachebanks
>  Qemu: Set cache banks
>  Resctrl: enable l3code/l3data
>  Resctrl: Make sure l3data/l3code are pairs
>  Resctrl: Compatible mode for cdp enabled
>  Resctrl: concurrence support
>  Resctrl: Scan resctrl before doing cache allocation
>
> docs/schemas/domaincommon.rng |   46 ++
> include/libvirt/virterror.h   |    1 +
> po/POTFILES.in                |    1 +
> src/Makefile.am               |    1 +
> src/conf/capabilities.c       |   56 ++
> src/conf/capabilities.h       |   23 +
> src/conf/domain_conf.c        |  182 +++++++
> src/conf/domain_conf.h        |   19 +
> src/libvirt_private.syms      |   11 +
> src/nodeinfo.c                |   64 +++
> src/nodeinfo.h                |    1 +
> src/qemu/qemu_capabilities.c  |    8 +
> src/qemu/qemu_driver.c        |    6 +
> src/qemu/qemu_process.c       |   53 ++
> src/util/virerror.c           |    1 +
> src/util/virresctrl.c         | 1208 +++++++++++++++++++++++++++++++++++++++++
> src/util/virresctrl.h         |   88 +++
> 17 files changed, 1769 insertions(+)
> create mode 100644 src/util/virresctrl.c
> create mode 100644 src/util/virresctrl.h
>
>--
>1.9.1
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list