[PATCH v3 RESEND for v7.4.0 00/14] Introduce virtio-mem <memory/> model

Michal Privoznik posted 14 patches 3 weeks, 3 days ago
Test syntax-check failed
Failed in applying to current master (apply log)
NEWS.rst                                      |  16 ++
docs/formatdomain.rst                         |  45 +++-
docs/kbase/index.rst                          |   4 +
docs/kbase/memorydevices.rst                  | 150 +++++++++++
docs/kbase/meson.build                        |   1 +
docs/manpages/virsh.rst                       |  30 +++
docs/schemas/domaincommon.rng                 |  16 ++
examples/c/misc/event-test.c                  |  17 ++
include/libvirt/libvirt-domain.h              |  23 ++
src/conf/domain_conf.c                        | 115 ++++++++-
src/conf/domain_conf.h                        |  15 ++
src/conf/domain_event.c                       |  84 +++++++
src/conf/domain_event.h                       |  10 +
src/conf/domain_validate.c                    |  39 +++
src/libvirt_private.syms                      |   5 +
src/qemu/qemu_alias.c                         |  10 +-
src/qemu/qemu_capabilities.c                  |   2 +
src/qemu/qemu_capabilities.h                  |   1 +
src/qemu/qemu_command.c                       |  13 +-
src/qemu/qemu_domain.c                        |  50 +++-
src/qemu/qemu_domain.h                        |   1 +
src/qemu/qemu_domain_address.c                |  38 ++-
src/qemu/qemu_driver.c                        | 233 +++++++++++++++++-
src/qemu/qemu_hotplug.c                       |  18 ++
src/qemu/qemu_hotplug.h                       |   5 +
src/qemu/qemu_monitor.c                       |  37 +++
src/qemu/qemu_monitor.h                       |  28 +++
src/qemu/qemu_monitor_json.c                  |  97 ++++++--
src/qemu/qemu_monitor_json.h                  |   5 +
src/qemu/qemu_process.c                       | 118 ++++++++-
src/qemu/qemu_validate.c                      |   8 +
src/remote/remote_daemon_dispatch.c           |  30 +++
src/remote/remote_driver.c                    |  32 +++
src/remote/remote_protocol.x                  |  14 +-
src/remote_protocol-structs                   |   7 +
src/security/security_apparmor.c              |   1 +
src/security/security_dac.c                   |   2 +
src/security/security_selinux.c               |   2 +
src/util/virhostmem.c                         |  63 +++++
src/util/virhostmem.h                         |   3 +
tests/domaincapsmock.c                        |   9 +
.../caps_5.1.0.x86_64.xml                     |   1 +
.../caps_5.2.0.x86_64.xml                     |   1 +
.../caps_6.0.0.x86_64.xml                     |   1 +
...mory-hotplug-virtio-mem.x86_64-latest.args |  41 +++
.../memory-hotplug-virtio-mem.xml             |  67 +++++
tests/qemuxml2argvtest.c                      |   1 +
...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
tests/qemuxml2xmltest.c                       |   1 +
tools/virsh-domain.c                          | 169 +++++++++++++
50 files changed, 1613 insertions(+), 67 deletions(-)
create mode 100644 docs/kbase/memorydevices.rst
create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml

[PATCH v3 RESEND for v7.4.0 00/14] Introduce virtio-mem <memory/> model

Posted by Michal Privoznik 3 weeks, 3 days ago
This is a rebased version of v3 I've sent about a month ago:

https://listman.redhat.com/archives/libvir-list/2021-March/msg00823.html

v2 here:

https://listman.redhat.com/archives/libvir-list/2021-February/msg00961.html

My intent is to merge these only after the upcoming release so that we
have the biggest test window possible.

diff to v2:
- Dropped code that forbade use of virtio-mem and memballoon at the same
  time;
- This meant that I had to adjust memory accounting,
  qemuDomainSetMemoryFlags() - see patches 11/15 and 12/15 which are new.
- Fixed small nits raised by Peter in his review of v2


Michal Prívozník (14):
  virhostmem: Introduce virHostMemGetTHPSize()
  qemu_process: Deduplicate code in qemuProcessNeedHugepagesPath()
  qemu_process: Drop needless check in
    qemuProcessNeedMemoryBackingPath()
  qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI
  conf: Introduce virtio-mem <memory/> model
  qemu: Build command line for virtio-mem
  qemu: Wire up <memory/> live update
  qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
  qemu: Refresh the actual size of virtio-mem on monitor reconnect
  qemu: Account for both memballoon and virtio-mem
  qemuDomainSetMemoryFlags: Take virtio-mem into consideration
  virsh: Introduce update-memory-device command
  news: document recent virtio memory addition
  kbase: Document virtio-mem

 NEWS.rst                                      |  16 ++
 docs/formatdomain.rst                         |  45 +++-
 docs/kbase/index.rst                          |   4 +
 docs/kbase/memorydevices.rst                  | 150 +++++++++++
 docs/kbase/meson.build                        |   1 +
 docs/manpages/virsh.rst                       |  30 +++
 docs/schemas/domaincommon.rng                 |  16 ++
 examples/c/misc/event-test.c                  |  17 ++
 include/libvirt/libvirt-domain.h              |  23 ++
 src/conf/domain_conf.c                        | 115 ++++++++-
 src/conf/domain_conf.h                        |  15 ++
 src/conf/domain_event.c                       |  84 +++++++
 src/conf/domain_event.h                       |  10 +
 src/conf/domain_validate.c                    |  39 +++
 src/libvirt_private.syms                      |   5 +
 src/qemu/qemu_alias.c                         |  10 +-
 src/qemu/qemu_capabilities.c                  |   2 +
 src/qemu/qemu_capabilities.h                  |   1 +
 src/qemu/qemu_command.c                       |  13 +-
 src/qemu/qemu_domain.c                        |  50 +++-
 src/qemu/qemu_domain.h                        |   1 +
 src/qemu/qemu_domain_address.c                |  38 ++-
 src/qemu/qemu_driver.c                        | 233 +++++++++++++++++-
 src/qemu/qemu_hotplug.c                       |  18 ++
 src/qemu/qemu_hotplug.h                       |   5 +
 src/qemu/qemu_monitor.c                       |  37 +++
 src/qemu/qemu_monitor.h                       |  28 +++
 src/qemu/qemu_monitor_json.c                  |  97 ++++++--
 src/qemu/qemu_monitor_json.h                  |   5 +
 src/qemu/qemu_process.c                       | 118 ++++++++-
 src/qemu/qemu_validate.c                      |   8 +
 src/remote/remote_daemon_dispatch.c           |  30 +++
 src/remote/remote_driver.c                    |  32 +++
 src/remote/remote_protocol.x                  |  14 +-
 src/remote_protocol-structs                   |   7 +
 src/security/security_apparmor.c              |   1 +
 src/security/security_dac.c                   |   2 +
 src/security/security_selinux.c               |   2 +
 src/util/virhostmem.c                         |  63 +++++
 src/util/virhostmem.h                         |   3 +
 tests/domaincapsmock.c                        |   9 +
 .../caps_5.1.0.x86_64.xml                     |   1 +
 .../caps_5.2.0.x86_64.xml                     |   1 +
 .../caps_6.0.0.x86_64.xml                     |   1 +
 ...mory-hotplug-virtio-mem.x86_64-latest.args |  41 +++
 .../memory-hotplug-virtio-mem.xml             |  67 +++++
 tests/qemuxml2argvtest.c                      |   1 +
 ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
 tests/qemuxml2xmltest.c                       |   1 +
 tools/virsh-domain.c                          | 169 +++++++++++++
 50 files changed, 1613 insertions(+), 67 deletions(-)
 create mode 100644 docs/kbase/memorydevices.rst
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
 create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml

-- 
2.26.3

Re: [PATCH v3 RESEND for v7.4.0 00/14] Introduce virtio-mem <memory/> model

Posted by Jing Qi 3 weeks ago
Tested with libvirt v7.2.0-381-g3c3c55be66 and
qemu-5.2.0-0.7.rc2.fc34.x86_64

S1.  Without hugepage
1.Start domain with virtio-mem device, and check the value of <memory> is
the total memory number including virtio-mem device.
# virsh start pc_test
2.  Hot plug a virtio-mem device, the values of memory & currentMemory are
refreshed accordingly.
 # cat mem.xml
    <memory model='virtio-mem'>
      <source>
        <nodemask>0</nodemask>
        <pagesize unit='KiB'>2048</pagesize>
      </source>
      <target>
        <size unit='KiB'>131072</size>
        <node>0</node>
        <block unit='KiB'>2048</block>
        <requested unit='KiB'>131072</requested>
        <actual unit='KiB'>131072</actual>
      </target>
    </memory>
#virsh attach-device pc_test mem.xml
3. Updated the requested size, and it's changed as expected from the output
of "virsh dumpxml "
# virsh update-memory-device pc_test --alias virtiomem1 --requested-size
100MiB

S2. With below hugepage configuration, tested above 3 steps again, and all
steps worked.
<memoryBacking>
 <hugepages/>
</memoryBacking>

S3. Hot plug virtio-pmem and the value of <memory> is updated accordingly.
  <memory model='virtio-pmem' access='shared'>
      <source>
        <path>/tmp/nvdimm</path>
      </source>
      <target>
        <size unit='KiB'>131072</size>
        <label>
        <size unit='KiB'>128</size>
        </label>
         </target>
    </memory>
# virsh attach-device pc_test pmem.xml
Device attached successfully
#virsh dumpxml pc_test
...

--
Tested-by Jing Qi <jinqi@redhat.com>



On Fri, Apr 23, 2021 at 9:25 PM Michal Privoznik <mprivozn@redhat.com>
wrote:

> This is a rebased version of v3 I've sent about a month ago:
>
> https://listman.redhat.com/archives/libvir-list/2021-March/msg00823.html
>
> v2 here:
>
> https://listman.redhat.com/archives/libvir-list/2021-February/msg00961.html
>
> My intent is to merge these only after the upcoming release so that we
> have the biggest test window possible.
>
> diff to v2:
> - Dropped code that forbade use of virtio-mem and memballoon at the same
>   time;
> - This meant that I had to adjust memory accounting,
>   qemuDomainSetMemoryFlags() - see patches 11/15 and 12/15 which are new.
> - Fixed small nits raised by Peter in his review of v2
>
>
> Michal Prívozník (14):
>   virhostmem: Introduce virHostMemGetTHPSize()
>   qemu_process: Deduplicate code in qemuProcessNeedHugepagesPath()
>   qemu_process: Drop needless check in
>     qemuProcessNeedMemoryBackingPath()
>   qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI
>   conf: Introduce virtio-mem <memory/> model
>   qemu: Build command line for virtio-mem
>   qemu: Wire up <memory/> live update
>   qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
>   qemu: Refresh the actual size of virtio-mem on monitor reconnect
>   qemu: Account for both memballoon and virtio-mem
>   qemuDomainSetMemoryFlags: Take virtio-mem into consideration
>   virsh: Introduce update-memory-device command
>   news: document recent virtio memory addition
>   kbase: Document virtio-mem
>
>  NEWS.rst                                      |  16 ++
>  docs/formatdomain.rst                         |  45 +++-
>  docs/kbase/index.rst                          |   4 +
>  docs/kbase/memorydevices.rst                  | 150 +++++++++++
>  docs/kbase/meson.build                        |   1 +
>  docs/manpages/virsh.rst                       |  30 +++
>  docs/schemas/domaincommon.rng                 |  16 ++
>  examples/c/misc/event-test.c                  |  17 ++
>  include/libvirt/libvirt-domain.h              |  23 ++
>  src/conf/domain_conf.c                        | 115 ++++++++-
>  src/conf/domain_conf.h                        |  15 ++
>  src/conf/domain_event.c                       |  84 +++++++
>  src/conf/domain_event.h                       |  10 +
>  src/conf/domain_validate.c                    |  39 +++
>  src/libvirt_private.syms                      |   5 +
>  src/qemu/qemu_alias.c                         |  10 +-
>  src/qemu/qemu_capabilities.c                  |   2 +
>  src/qemu/qemu_capabilities.h                  |   1 +
>  src/qemu/qemu_command.c                       |  13 +-
>  src/qemu/qemu_domain.c                        |  50 +++-
>  src/qemu/qemu_domain.h                        |   1 +
>  src/qemu/qemu_domain_address.c                |  38 ++-
>  src/qemu/qemu_driver.c                        | 233 +++++++++++++++++-
>  src/qemu/qemu_hotplug.c                       |  18 ++
>  src/qemu/qemu_hotplug.h                       |   5 +
>  src/qemu/qemu_monitor.c                       |  37 +++
>  src/qemu/qemu_monitor.h                       |  28 +++
>  src/qemu/qemu_monitor_json.c                  |  97 ++++++--
>  src/qemu/qemu_monitor_json.h                  |   5 +
>  src/qemu/qemu_process.c                       | 118 ++++++++-
>  src/qemu/qemu_validate.c                      |   8 +
>  src/remote/remote_daemon_dispatch.c           |  30 +++
>  src/remote/remote_driver.c                    |  32 +++
>  src/remote/remote_protocol.x                  |  14 +-
>  src/remote_protocol-structs                   |   7 +
>  src/security/security_apparmor.c              |   1 +
>  src/security/security_dac.c                   |   2 +
>  src/security/security_selinux.c               |   2 +
>  src/util/virhostmem.c                         |  63 +++++
>  src/util/virhostmem.h                         |   3 +
>  tests/domaincapsmock.c                        |   9 +
>  .../caps_5.1.0.x86_64.xml                     |   1 +
>  .../caps_5.2.0.x86_64.xml                     |   1 +
>  .../caps_6.0.0.x86_64.xml                     |   1 +
>  ...mory-hotplug-virtio-mem.x86_64-latest.args |  41 +++
>  .../memory-hotplug-virtio-mem.xml             |  67 +++++
>  tests/qemuxml2argvtest.c                      |   1 +
>  ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
>  tests/qemuxml2xmltest.c                       |   1 +
>  tools/virsh-domain.c                          | 169 +++++++++++++
>  50 files changed, 1613 insertions(+), 67 deletions(-)
>  create mode 100644 docs/kbase/memorydevices.rst
>  create mode 100644
> tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
>  create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
>  create mode 120000
> tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml
>
> --
> 2.26.3
>
>

-- 
Thanks & Regards,
Jing,Qi