[PATCH for 7.0.0 v1 00/26] Introduce virtio memory support

Michal Privoznik posted 26 patches 3 years, 5 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/cover.1606489221.git.mprivozn@redhat.com
There is a newer version of this series
docs/formatdomain.rst                         |  70 +++-
docs/schemas/domaincommon.rng                 |  16 +
src/conf/domain_conf.c                        | 372 ++++++++++++++----
src/conf/domain_conf.h                        |  38 +-
src/internal.h                                |  10 +
src/libvirt_private.syms                      |   2 +
src/qemu/qemu_alias.c                         |  59 ++-
src/qemu/qemu_capabilities.c                  |   4 +
src/qemu/qemu_capabilities.h                  |   2 +
src/qemu/qemu_cgroup.c                        |  43 +-
src/qemu/qemu_command.c                       | 172 +++++---
src/qemu/qemu_command.h                       |   5 +-
src/qemu/qemu_domain.c                        |  99 +++--
src/qemu/qemu_domain.h                        |   1 +
src/qemu/qemu_domain_address.c                |  98 +++--
src/qemu/qemu_domain_address.h                |   3 +-
src/qemu/qemu_driver.c                        | 198 +++++++++-
src/qemu/qemu_hotplug.c                       |  22 +-
src/qemu/qemu_hotplug.h                       |   5 +
src/qemu/qemu_monitor.c                       |  37 ++
src/qemu/qemu_monitor.h                       |  27 ++
src/qemu/qemu_monitor_json.c                  |  94 +++--
src/qemu/qemu_monitor_json.h                  |   5 +
src/qemu/qemu_namespace.c                     |  19 +-
src/qemu/qemu_process.c                       |  52 ++-
src/qemu/qemu_validate.c                      |  78 ++--
src/security/security_apparmor.c              |  35 +-
src/security/security_dac.c                   |  48 ++-
src/security/security_selinux.c               |  48 ++-
src/security/virt-aa-helper.c                 |  22 +-
src/util/virrandom.c                          |   2 +-
src/util/viruuid.c                            |  17 +-
src/util/viruuid.h                            |   2 +-
.../caps_4.1.0.x86_64.xml                     |   1 +
.../caps_4.2.0.x86_64.xml                     |   1 +
.../caps_5.0.0.x86_64.xml                     |   1 +
.../caps_5.1.0.x86_64.xml                     |   2 +
.../caps_5.2.0.x86_64.xml                     |   2 +
...mory-hotplug-virtio-mem.x86_64-latest.args |  53 +++
.../memory-hotplug-virtio-mem.xml             |  78 ++++
...ory-hotplug-virtio-pmem.x86_64-latest.args |  45 +++
.../memory-hotplug-virtio-pmem.xml            |  54 +++
tests/qemuxml2argvtest.c                      |   2 +
...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
...mory-hotplug-virtio-pmem.x86_64-latest.xml |   1 +
tests/qemuxml2xmltest.c                       |   3 +
tools/virsh-domain.c                          | 138 ++++++-
47 files changed, 1703 insertions(+), 384 deletions(-)
create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.xml
create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml
create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-pmem.x86_64-latest.xml
[PATCH for 7.0.0 v1 00/26] Introduce virtio memory support
Posted by Michal Privoznik 3 years, 5 months ago
Available also here:

https://gitlab.com/MichalPrivoznik/libvirt/-/commits/virtio_mem/

There are new virtio variants of pc-dimm and nvdimm devices. This is the
first attempt to impalement support for them in libvirt.

Thanks to David Hildenbrand for his valuable input!

Michal Prívozník (26):
  viruuid: Rework virUUIDIsValid()
  internal.h: Introduce VIR_IS_POW2()
  docs: Fix nvdimm example wrt to <uuid/>
  domain_conf: Check NVDIMM UUID in ABI stability
  qemu_domain_address: Reformat qemuDomainAssignS390Addresses()
  conf: Require nvdimm path in validate step
  domain_conf: Fix virDomainMemoryModel type
  virDomainMemorySourceDefFormat: Utilize virXMLFormatElement()
  virDomainMemoryTargetDefFormat: Utilize virXMLFormatElement()
  qemu: Move mem validation into post parse validator
  conf: Move some of virDomainMemoryDef members into a union
  conf: Introduce virtio-pmem <memory/> model
  qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_{P}MEM_PCI
  qemu_validate: Require virtio-mem device for mem model virtio
  security: Relabel virtio mem
  qemu: Allow virtio-pmem in CGroups
  qemu: Create virtio-pmem in domain namespace
  qemu_command: Move dimm into qemuBuildDeviceAddressStr()
  qemu: Build command line for virtio-pmem
  conf: Introduce virtio-mem <memory/> model
  qemu: Build command line for virtio-mem
  qemu: Wire up <memory/> live update
  qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
  qemu: Refresh the actual size of virtio-mem on monitor reconnect
  virsh: Simplify @flags handing in cmdSetmem() and cmdSetmaxmem()
  virsh: Introduce --virtio to setmem

 docs/formatdomain.rst                         |  70 +++-
 docs/schemas/domaincommon.rng                 |  16 +
 src/conf/domain_conf.c                        | 372 ++++++++++++++----
 src/conf/domain_conf.h                        |  38 +-
 src/internal.h                                |  10 +
 src/libvirt_private.syms                      |   2 +
 src/qemu/qemu_alias.c                         |  59 ++-
 src/qemu/qemu_capabilities.c                  |   4 +
 src/qemu/qemu_capabilities.h                  |   2 +
 src/qemu/qemu_cgroup.c                        |  43 +-
 src/qemu/qemu_command.c                       | 172 +++++---
 src/qemu/qemu_command.h                       |   5 +-
 src/qemu/qemu_domain.c                        |  99 +++--
 src/qemu/qemu_domain.h                        |   1 +
 src/qemu/qemu_domain_address.c                |  98 +++--
 src/qemu/qemu_domain_address.h                |   3 +-
 src/qemu/qemu_driver.c                        | 198 +++++++++-
 src/qemu/qemu_hotplug.c                       |  22 +-
 src/qemu/qemu_hotplug.h                       |   5 +
 src/qemu/qemu_monitor.c                       |  37 ++
 src/qemu/qemu_monitor.h                       |  27 ++
 src/qemu/qemu_monitor_json.c                  |  94 +++--
 src/qemu/qemu_monitor_json.h                  |   5 +
 src/qemu/qemu_namespace.c                     |  19 +-
 src/qemu/qemu_process.c                       |  52 ++-
 src/qemu/qemu_validate.c                      |  78 ++--
 src/security/security_apparmor.c              |  35 +-
 src/security/security_dac.c                   |  48 ++-
 src/security/security_selinux.c               |  48 ++-
 src/security/virt-aa-helper.c                 |  22 +-
 src/util/virrandom.c                          |   2 +-
 src/util/viruuid.c                            |  17 +-
 src/util/viruuid.h                            |   2 +-
 .../caps_4.1.0.x86_64.xml                     |   1 +
 .../caps_4.2.0.x86_64.xml                     |   1 +
 .../caps_5.0.0.x86_64.xml                     |   1 +
 .../caps_5.1.0.x86_64.xml                     |   2 +
 .../caps_5.2.0.x86_64.xml                     |   2 +
 ...mory-hotplug-virtio-mem.x86_64-latest.args |  53 +++
 .../memory-hotplug-virtio-mem.xml             |  78 ++++
 ...ory-hotplug-virtio-pmem.x86_64-latest.args |  45 +++
 .../memory-hotplug-virtio-pmem.xml            |  54 +++
 tests/qemuxml2argvtest.c                      |   2 +
 ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
 ...mory-hotplug-virtio-pmem.x86_64-latest.xml |   1 +
 tests/qemuxml2xmltest.c                       |   3 +
 tools/virsh-domain.c                          | 138 ++++++-
 47 files changed, 1703 insertions(+), 384 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.xml
 create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml
 create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-pmem.x86_64-latest.xml

-- 
2.26.2

Re: [PATCH for 7.0.0 v1 00/26] Introduce virtio memory support
Posted by Daniel Henrique Barboza 3 years, 5 months ago

On 11/27/20 12:02 PM, Michal Privoznik wrote:
> Available also here:
> 
> https://gitlab.com/MichalPrivoznik/libvirt/-/commits/virtio_mem/
> 
> There are new virtio variants of pc-dimm and nvdimm devices. This is the
> first attempt to impalement support for them in libvirt.
> 
> Thanks to David Hildenbrand for his valuable input!

Series LGTM:

Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>


I did a field test in my x86_64 (since pSeries does not support neither
virtio-mem nor virtio-pmem), played around with 'virsh setmem --virtio'
and, as a non-x86 expert, looks good to me as well:


Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>

> 
> Michal Prívozník (26):
>    viruuid: Rework virUUIDIsValid()
>    internal.h: Introduce VIR_IS_POW2()
>    docs: Fix nvdimm example wrt to <uuid/>
>    domain_conf: Check NVDIMM UUID in ABI stability
>    qemu_domain_address: Reformat qemuDomainAssignS390Addresses()
>    conf: Require nvdimm path in validate step
>    domain_conf: Fix virDomainMemoryModel type
>    virDomainMemorySourceDefFormat: Utilize virXMLFormatElement()
>    virDomainMemoryTargetDefFormat: Utilize virXMLFormatElement()
>    qemu: Move mem validation into post parse validator
>    conf: Move some of virDomainMemoryDef members into a union
>    conf: Introduce virtio-pmem <memory/> model
>    qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_{P}MEM_PCI
>    qemu_validate: Require virtio-mem device for mem model virtio
>    security: Relabel virtio mem
>    qemu: Allow virtio-pmem in CGroups
>    qemu: Create virtio-pmem in domain namespace
>    qemu_command: Move dimm into qemuBuildDeviceAddressStr()
>    qemu: Build command line for virtio-pmem
>    conf: Introduce virtio-mem <memory/> model
>    qemu: Build command line for virtio-mem
>    qemu: Wire up <memory/> live update
>    qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
>    qemu: Refresh the actual size of virtio-mem on monitor reconnect
>    virsh: Simplify @flags handing in cmdSetmem() and cmdSetmaxmem()
>    virsh: Introduce --virtio to setmem
> 
>   docs/formatdomain.rst                         |  70 +++-
>   docs/schemas/domaincommon.rng                 |  16 +
>   src/conf/domain_conf.c                        | 372 ++++++++++++++----
>   src/conf/domain_conf.h                        |  38 +-
>   src/internal.h                                |  10 +
>   src/libvirt_private.syms                      |   2 +
>   src/qemu/qemu_alias.c                         |  59 ++-
>   src/qemu/qemu_capabilities.c                  |   4 +
>   src/qemu/qemu_capabilities.h                  |   2 +
>   src/qemu/qemu_cgroup.c                        |  43 +-
>   src/qemu/qemu_command.c                       | 172 +++++---
>   src/qemu/qemu_command.h                       |   5 +-
>   src/qemu/qemu_domain.c                        |  99 +++--
>   src/qemu/qemu_domain.h                        |   1 +
>   src/qemu/qemu_domain_address.c                |  98 +++--
>   src/qemu/qemu_domain_address.h                |   3 +-
>   src/qemu/qemu_driver.c                        | 198 +++++++++-
>   src/qemu/qemu_hotplug.c                       |  22 +-
>   src/qemu/qemu_hotplug.h                       |   5 +
>   src/qemu/qemu_monitor.c                       |  37 ++
>   src/qemu/qemu_monitor.h                       |  27 ++
>   src/qemu/qemu_monitor_json.c                  |  94 +++--
>   src/qemu/qemu_monitor_json.h                  |   5 +
>   src/qemu/qemu_namespace.c                     |  19 +-
>   src/qemu/qemu_process.c                       |  52 ++-
>   src/qemu/qemu_validate.c                      |  78 ++--
>   src/security/security_apparmor.c              |  35 +-
>   src/security/security_dac.c                   |  48 ++-
>   src/security/security_selinux.c               |  48 ++-
>   src/security/virt-aa-helper.c                 |  22 +-
>   src/util/virrandom.c                          |   2 +-
>   src/util/viruuid.c                            |  17 +-
>   src/util/viruuid.h                            |   2 +-
>   .../caps_4.1.0.x86_64.xml                     |   1 +
>   .../caps_4.2.0.x86_64.xml                     |   1 +
>   .../caps_5.0.0.x86_64.xml                     |   1 +
>   .../caps_5.1.0.x86_64.xml                     |   2 +
>   .../caps_5.2.0.x86_64.xml                     |   2 +
>   ...mory-hotplug-virtio-mem.x86_64-latest.args |  53 +++
>   .../memory-hotplug-virtio-mem.xml             |  78 ++++
>   ...ory-hotplug-virtio-pmem.x86_64-latest.args |  45 +++
>   .../memory-hotplug-virtio-pmem.xml            |  54 +++
>   tests/qemuxml2argvtest.c                      |   2 +
>   ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
>   ...mory-hotplug-virtio-pmem.x86_64-latest.xml |   1 +
>   tests/qemuxml2xmltest.c                       |   3 +
>   tools/virsh-domain.c                          | 138 ++++++-
>   47 files changed, 1703 insertions(+), 384 deletions(-)
>   create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
>   create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
>   create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.x86_64-latest.args
>   create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.xml
>   create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml
>   create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-pmem.x86_64-latest.xml
> 

Re: [PATCH for 7.0.0 v1 00/26] Introduce virtio memory support
Posted by Han Han 3 years, 4 months ago
Works for me on qemu-5.2.0-0.7.rc2.fc34.x86_64, libvirt
v6.10.0-61-gd8c9cc3bd0 with this series patched:
guest kernel: kernel-5.10.0-0.rc5.20201125git127c501a03d5.85.fc33.x86_64
recompiled with VIRTIO_PMEM

Steps:
1. Start VM with <maxMemory> and <numa>:
Domain xml:
<domain type='kvm' id='5'>
  <name>pc</name>
  <uuid>bb508b28-d57b-44bd-9e9c-a134cef24b60</uuid>
  <maxMemory slots='16' unit='KiB'>20972544</maxMemory>
  <memory unit='KiB'>1048576</memory>
  <cpu mode='custom' match='exact' check='full'>
    <numa>
      <cell id='0' cpus='0-3' memory='1048576' unit='KiB'/>
    </numa>
  </cpu>
...
</domain>

2. Attach the virtio memory device:
➜  ~ cat /tmp/virtio-mem.xml
  <memory model='virtio'>
    <target>
      <size unit='KiB'>4194304</size>
      <node>0</node>
      <block unit='KiB'>2048</block>
      <requested unit='KiB'>524288</requested>
    </target>
  </memory>

➜  ~ virsh attach-device pc /tmp/virtio-mem.xml
Device attached successfully

Before memory attached, the VM memory is:
[root@localhost ~]# free -m
              total        used        free      shared  buff/cache
available
Mem:            879         124         601           2         152
615
Swap:             0           0           0

After:
[root@localhost ~]# free -m
              total        used        free      shared  buff/cache
available
Mem:           1391         132        1106           2         153
 1074
Swap:             0           0           0


3. Attach the virtio pmem device:
➜  ~ cat /tmp/virtio-pmem.xml
<memory model='virtio' access='shared'>
    <source>
      <path>/tmp/virtio_pmem</path>
      <pmem/>
    </source>
    <target>
      <size unit='KiB'>524288</size>
    </target>
  </memory>

➜  ~ virsh attach-device pc /tmp/virtio-pmem.xml
Device attached successfully

Check it in VM:
[root@localhost ~]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    252:0    0   10G  0 disk
└─vda1 252:1    0   10G  0 part /
pmem0  259:0    0  512M  0 disk

3. Try to detach them:
➜  ~ virsh detach-device pc /tmp/virtio-mem.xml
error: Failed to detach device from /tmp/virtio-mem.xml
error: internal error: unable to execute QEMU command 'device_del': virtio
based memory devices cannot be unplugged.

➜  ~ virsh detach-device pc /tmp/virtio-pmem.xml
error: Failed to detach device from /tmp/virtio-pmem.xml
error: internal error: unable to execute QEMU command 'device_del': virtio
based memory devices cannot be unplugged.

On Fri, Nov 27, 2020 at 11:05 PM Michal Privoznik <mprivozn@redhat.com>
wrote:

> Available also here:
>
> https://gitlab.com/MichalPrivoznik/libvirt/-/commits/virtio_mem/
>
> There are new virtio variants of pc-dimm and nvdimm devices. This is the
> first attempt to impalement support for them in libvirt.
>
> Thanks to David Hildenbrand for his valuable input!
>
> Michal Prívozník (26):
>   viruuid: Rework virUUIDIsValid()
>   internal.h: Introduce VIR_IS_POW2()
>   docs: Fix nvdimm example wrt to <uuid/>
>   domain_conf: Check NVDIMM UUID in ABI stability
>   qemu_domain_address: Reformat qemuDomainAssignS390Addresses()
>   conf: Require nvdimm path in validate step
>   domain_conf: Fix virDomainMemoryModel type
>   virDomainMemorySourceDefFormat: Utilize virXMLFormatElement()
>   virDomainMemoryTargetDefFormat: Utilize virXMLFormatElement()
>   qemu: Move mem validation into post parse validator
>   conf: Move some of virDomainMemoryDef members into a union
>   conf: Introduce virtio-pmem <memory/> model
>   qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_{P}MEM_PCI
>   qemu_validate: Require virtio-mem device for mem model virtio
>   security: Relabel virtio mem
>   qemu: Allow virtio-pmem in CGroups
>   qemu: Create virtio-pmem in domain namespace
>   qemu_command: Move dimm into qemuBuildDeviceAddressStr()
>   qemu: Build command line for virtio-pmem
>   conf: Introduce virtio-mem <memory/> model
>   qemu: Build command line for virtio-mem
>   qemu: Wire up <memory/> live update
>   qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
>   qemu: Refresh the actual size of virtio-mem on monitor reconnect
>   virsh: Simplify @flags handing in cmdSetmem() and cmdSetmaxmem()
>   virsh: Introduce --virtio to setmem
>
>  docs/formatdomain.rst                         |  70 +++-
>  docs/schemas/domaincommon.rng                 |  16 +
>  src/conf/domain_conf.c                        | 372 ++++++++++++++----
>  src/conf/domain_conf.h                        |  38 +-
>  src/internal.h                                |  10 +
>  src/libvirt_private.syms                      |   2 +
>  src/qemu/qemu_alias.c                         |  59 ++-
>  src/qemu/qemu_capabilities.c                  |   4 +
>  src/qemu/qemu_capabilities.h                  |   2 +
>  src/qemu/qemu_cgroup.c                        |  43 +-
>  src/qemu/qemu_command.c                       | 172 +++++---
>  src/qemu/qemu_command.h                       |   5 +-
>  src/qemu/qemu_domain.c                        |  99 +++--
>  src/qemu/qemu_domain.h                        |   1 +
>  src/qemu/qemu_domain_address.c                |  98 +++--
>  src/qemu/qemu_domain_address.h                |   3 +-
>  src/qemu/qemu_driver.c                        | 198 +++++++++-
>  src/qemu/qemu_hotplug.c                       |  22 +-
>  src/qemu/qemu_hotplug.h                       |   5 +
>  src/qemu/qemu_monitor.c                       |  37 ++
>  src/qemu/qemu_monitor.h                       |  27 ++
>  src/qemu/qemu_monitor_json.c                  |  94 +++--
>  src/qemu/qemu_monitor_json.h                  |   5 +
>  src/qemu/qemu_namespace.c                     |  19 +-
>  src/qemu/qemu_process.c                       |  52 ++-
>  src/qemu/qemu_validate.c                      |  78 ++--
>  src/security/security_apparmor.c              |  35 +-
>  src/security/security_dac.c                   |  48 ++-
>  src/security/security_selinux.c               |  48 ++-
>  src/security/virt-aa-helper.c                 |  22 +-
>  src/util/virrandom.c                          |   2 +-
>  src/util/viruuid.c                            |  17 +-
>  src/util/viruuid.h                            |   2 +-
>  .../caps_4.1.0.x86_64.xml                     |   1 +
>  .../caps_4.2.0.x86_64.xml                     |   1 +
>  .../caps_5.0.0.x86_64.xml                     |   1 +
>  .../caps_5.1.0.x86_64.xml                     |   2 +
>  .../caps_5.2.0.x86_64.xml                     |   2 +
>  ...mory-hotplug-virtio-mem.x86_64-latest.args |  53 +++
>  .../memory-hotplug-virtio-mem.xml             |  78 ++++
>  ...ory-hotplug-virtio-pmem.x86_64-latest.args |  45 +++
>  .../memory-hotplug-virtio-pmem.xml            |  54 +++
>  tests/qemuxml2argvtest.c                      |   2 +
>  ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
>  ...mory-hotplug-virtio-pmem.x86_64-latest.xml |   1 +
>  tests/qemuxml2xmltest.c                       |   3 +
>  tools/virsh-domain.c                          | 138 ++++++-
>  47 files changed, 1703 insertions(+), 384 deletions(-)
>  create mode 100644
> tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
>  create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
>  create mode 100644
> tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.x86_64-latest.args
>  create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.xml
>  create mode 120000
> tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml
>  create mode 120000
> tests/qemuxml2xmloutdata/memory-hotplug-virtio-pmem.x86_64-latest.xml
>
> --
> 2.26.2
>
>

-- 
Tested-by: Han Han <hhan@redhat.com>