qemu: Don't prealloc mem for real NVDIMMs

[PATCH v2] qemu: Don't prealloc mem for real NVDIMMs

Posted by Michal Privoznik 3 years, 3 months ago

Currently, we configure QEMU to prealloc memory almost by
default. Well, by default for NVDIMMs, hugepages and if user
asked us to (via memoryBacking <allocation mode="immediate"/>).

However, when guest's NVDIMM is backed by real life NVDIMM this
approach is not the best. In this case users should put <pmem/>
into the <memory/> device <source/>, like this:

  <memory model='nvdimm' access='shared'>
    <source>
      <path>/dev/pmem0</path>
      <pmem/>
    </source>
  </memory>

Instructing QEMU to do prealloc in this case means that each
page of the NVDIMM is "touched" (the first byte is read and
written back - see QEMU commit v2.9.0-rc1~26^2) which cripples
device wear.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1894053
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
---

v2 of:

https://www.redhat.com/archives/libvir-list/2020-November/msg01568.html

diff to v1:
- I've dropped the first hunk of v1 which forbade prealloc even if user
  requested it explicitly.

 src/qemu/qemu_command.c                                      | 5 ++++-
 .../memory-hotplug-nvdimm-pmem.x86_64-latest.args            | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index b06a086e18..f572ed64c1 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -3085,7 +3085,10 @@ qemuBuildMemoryBackendProps(virJSONValuePtr *backendProps,
 
         if (mem->nvdimmPath) {
             memPath = g_strdup(mem->nvdimmPath);
-            prealloc = true;
+            /* If the NVDIMM is a real device then there's nothing to prealloc.
+             * If anyhing, we would be only wearing off the device. */
+            if (!mem->nvdimmPmem)
+                prealloc = true;
         } else if (useHugepage) {
             if (qemuGetDomainHupageMemPath(priv->driver, def, pagesize, &memPath) < 0)
                 return -1;
diff --git a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args
index cac02a6f6d..fb4ae4b518 100644
--- a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args
@@ -20,7 +20,7 @@ file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
 -object memory-backend-ram,id=ram-node0,size=224395264 \
 -numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \
 -object memory-backend-file,id=memnvdimm0,mem-path=/tmp/nvdimm,share=no,\
-prealloc=yes,size=536870912,pmem=yes \
+size=536870912,pmem=yes \
 -device nvdimm,node=0,memdev=memnvdimm0,id=nvdimm0,slot=0 \
 -uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
 -display none \
-- 
2.26.2

Re: [PATCH v2] qemu: Don't prealloc mem for real NVDIMMs

Posted by Daniel Henrique Barboza 3 years, 3 months ago


On 1/5/21 8:39 AM, Michal Privoznik wrote:
> Currently, we configure QEMU to prealloc memory almost by
> default. Well, by default for NVDIMMs, hugepages and if user
> asked us to (via memoryBacking <allocation mode="immediate"/>).
> 
> However, when guest's NVDIMM is backed by real life NVDIMM this
> approach is not the best. In this case users should put <pmem/>
> into the <memory/> device <source/>, like this:
> 
>    <memory model='nvdimm' access='shared'>
>      <source>
>        <path>/dev/pmem0</path>
>        <pmem/>
>      </source>
>    </memory>
> 
> Instructing QEMU to do prealloc in this case means that each
> page of the NVDIMM is "touched" (the first byte is read and
> written back - see QEMU commit v2.9.0-rc1~26^2) which cripples
> device wear.
> 
> Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1894053
> Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
> ---
> 
> v2 of:
> 
> https://www.redhat.com/archives/libvir-list/2020-November/msg01568.html
> 
> diff to v1:
> - I've dropped the first hunk of v1 which forbade prealloc even if user
>    requested it explicitly.
> 
>   src/qemu/qemu_command.c                                      | 5 ++++-
>   .../memory-hotplug-nvdimm-pmem.x86_64-latest.args            | 2 +-
>   2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
> index b06a086e18..f572ed64c1 100644
> --- a/src/qemu/qemu_command.c
> +++ b/src/qemu/qemu_command.c
> @@ -3085,7 +3085,10 @@ qemuBuildMemoryBackendProps(virJSONValuePtr *backendProps,
>   
>           if (mem->nvdimmPath) {
>               memPath = g_strdup(mem->nvdimmPath);
> -            prealloc = true;
> +            /* If the NVDIMM is a real device then there's nothing to prealloc.
> +             * If anyhing, we would be only wearing off the device. */

s/anyhing/anything

Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>

> +            if (!mem->nvdimmPmem)
> +                prealloc = true;
>           } else if (useHugepage) {
>               if (qemuGetDomainHupageMemPath(priv->driver, def, pagesize, &memPath) < 0)
>                   return -1;
> diff --git a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args
> index cac02a6f6d..fb4ae4b518 100644
> --- a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args
> +++ b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args
> @@ -20,7 +20,7 @@ file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
>   -object memory-backend-ram,id=ram-node0,size=224395264 \
>   -numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \
>   -object memory-backend-file,id=memnvdimm0,mem-path=/tmp/nvdimm,share=no,\
> -prealloc=yes,size=536870912,pmem=yes \
> +size=536870912,pmem=yes \
>   -device nvdimm,node=0,memdev=memnvdimm0,id=nvdimm0,slot=0 \
>   -uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
>   -display none \
>

Re: [PATCH v2] qemu: Don't prealloc mem for real NVDIMMs

Posted by Daniel P. Berrangé 3 years, 3 months ago

On Tue, Jan 05, 2021 at 12:39:00PM +0100, Michal Privoznik wrote:
> Currently, we configure QEMU to prealloc memory almost by
> default. Well, by default for NVDIMMs, hugepages and if user
> asked us to (via memoryBacking <allocation mode="immediate"/>).
> 
> However, when guest's NVDIMM is backed by real life NVDIMM this
> approach is not the best. In this case users should put <pmem/>
> into the <memory/> device <source/>, like this:
> 
>   <memory model='nvdimm' access='shared'>
>     <source>
>       <path>/dev/pmem0</path>
>       <pmem/>
>     </source>
>   </memory>
> 
> Instructing QEMU to do prealloc in this case means that each
> page of the NVDIMM is "touched" (the first byte is read and
> written back - see QEMU commit v2.9.0-rc1~26^2) which cripples
> device wear.
> 
> Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1894053
> Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
> ---
> 
> v2 of:
> 
> https://www.redhat.com/archives/libvir-list/2020-November/msg01568.html
> 
> diff to v1:
> - I've dropped the first hunk of v1 which forbade prealloc even if user
>   requested it explicitly.
> 
>  src/qemu/qemu_command.c                                      | 5 ++++-
>  .../memory-hotplug-nvdimm-pmem.x86_64-latest.args            | 2 +-
>  2 files changed, 5 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|