[Qemu-devel] [PATCH 0/7 V11] nvdimm: guarantee persistence of QEMU writes to persistent memory

junyan.he@gmx.com posted 7 patches 5 years, 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/1531900083-18167-1-git-send-email-junyan.he@gmx.com
Test docker-mingw@fedora passed
Test checkpatch passed
Test docker-quick@centos7 passed
Test docker-clang@ubuntu passed
backends/hostmem-file.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
configure               | 29 +++++++++++++++++++++++++++++
docs/nvdimm.txt         | 22 ++++++++++++++++++++++
exec.c                  | 38 +++++++++++++-------------------------
hw/mem/nvdimm.c         |  9 ++++++++-
include/exec/memory.h   | 31 +++++++++++++++++++++++++++++--
include/exec/ram_addr.h | 28 ++++++++++++++++++++++++++--
include/qemu/pmem.h     | 36 ++++++++++++++++++++++++++++++++++++
memory.c                |  8 +++++---
migration/ram.c         | 17 +++++++++++++++++
numa.c                  |  2 +-
qemu-options.hx         |  7 +++++++
12 files changed, 235 insertions(+), 36 deletions(-)
create mode 100644 include/qemu/pmem.h
[Qemu-devel] [PATCH 0/7 V11] nvdimm: guarantee persistence of QEMU writes to persistent memory
Posted by junyan.he@gmx.com 5 years, 8 months ago
From: Junyan He <junyan.he@intel.com>

QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration.
If the backend is on the persistent memory, QEMU needs to take proper operations to
ensure its writes persistent on the persistent memory. Otherwise, a host power failure
may result in the loss the guest data on the persistent memory.

This patch series is based on Marcel's patch "mem: add share parameter to memory-backend-ram" [1]
because of the changes in patch 1.
[1] https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg03858.html

Previous versions of this patch series can be found at: 
v10: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg03433.html
v9: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg02361.html
v8: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg02279.html
v7: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg02997.html
v6: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg00061.html
v5: https://lists.gnu.org/archive/html/qemu-devel/2018-05/msg02258.html
V4: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg06993.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg04365.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg01579.html
v1: https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg05040.html

Changes in v11:
* (Patch 2) Modify the ram_flags parameter to 32bits, the same size as it in RAMBlock
* (Patch 5 and Patch 7) Delete pmem_xxx stub functions in stubs/pmem.c. Use inline
functions with assert to replace them, because we never come there when pmem is enabled
but lack of libpmem support.

Changes in v10:
* (Patch 4) Fix a nit in nvdimm docs about pmem option usage in command line 
The v10 patch set is all reviewed by Igor Mammedov <imammedo@redhat.com>

Changes in v9:
* (Patch 3 and Patch 4) Reorder these two patches to make logic right.
Firstly add libpmem support, and then we can use libpmem's configure
check result. Also fix some typo and grammar issues in these two patches.

Changs in v8: 
* (Patch 3) Report a error when user set 'pmem' to file-backend, while
the qemu is lack of libpmem support. In this case, we can not ensure
the persistence of the file-backend, so we choose to fail the build
rather than contine and make the thing more confused.

Changes in v7: 
The v6 patch set has already reviewed by Stefan Hajnoczi <stefanha@redhat.com>
No logic change in this v7 version, just:
* Spelling check and some document words refined.
* Rebase to "ram is migratable" patch set.

Changes in v6: 
* (Patch 1) Expose all ram block flags rather than redefine the flags.
* (Patch 4) Use pkg-config rather the hard check when configure. 
* (Patch 7) Sync and flush all the pmem data when migration completes,
rather than sync pages one by one in previous version.

Changes in v5: 
* (Patch 9) Add post copy check and output some messages for nvdimm.

Changes in v4: 
* (Patch 2) Fix compilation errors found by patchew.

Changes in v3:
* (Patch 5) Add a is_pmem flag to ram_handle_compressed() and handle
PMEM writes in it, so we don't need the _common function.
* (Patch 6) Expose qemu_get_buffer_common so we can remove the
unnecessary qemu_get_buffer_to_pmem wrapper.
* (Patch 8) Add a is_pmem flag to xbzrle_decode_buffer() and handle
PMEM writes in it, so we can remove the unnecessary
xbzrle_decode_buffer_{common, to_pmem}.
* Move libpmem stubs to stubs/pmem.c and fix the compilation failures
of test-{xbzrle,vmstate}.c.

Changes in v2:
* (Patch 1) Use a flags parameter in file ram allocation functions.
* (Patch 2) Add a new option 'pmem' to hostmem-file.
* (Patch 3) Use libpmem to operate on the persistent memory, rather
than re-implementing those operations in QEMU.
* (Patch 5-8) Consider the write persistence in the migration path.


Junyan:
[1/7] memory, exec: Expose all memory block related flags.
[6/7] migration/ram: Add check and info message to nvdimm post copy.
[7/7] migration/ram: ensure write persistence on loading all date to PMEM.

Haozhong:
[5/7] mem/nvdimm: ensure write persistence to PMEM in label emulation

Haozhong & Junyan:
[2/7] memory, exec: switch file ram allocation functions to 'flags' parameters
[3/7] configure: add libpmem support [4/7] hostmem-file: add the 'pmem' option
[4/7] hostmem-file: add the 'pmem' option

--
backends/hostmem-file.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
configure               | 29 +++++++++++++++++++++++++++++
docs/nvdimm.txt         | 22 ++++++++++++++++++++++
exec.c                  | 38 +++++++++++++-------------------------
hw/mem/nvdimm.c         |  9 ++++++++-
include/exec/memory.h   | 31 +++++++++++++++++++++++++++++--
include/exec/ram_addr.h | 28 ++++++++++++++++++++++++++--
include/qemu/pmem.h     | 36 ++++++++++++++++++++++++++++++++++++
memory.c                |  8 +++++---
migration/ram.c         | 17 +++++++++++++++++
numa.c                  |  2 +-
qemu-options.hx         |  7 +++++++
12 files changed, 235 insertions(+), 36 deletions(-)
create mode 100644 include/qemu/pmem.h

-- 
2.7.4


Re: [Qemu-devel] [PATCH 0/7 V11] nvdimm: guarantee persistence of QEMU writes to persistent memory
Posted by Stefan Hajnoczi 5 years, 8 months ago
On Wed, Jul 18, 2018 at 03:47:56PM +0800, junyan.he@gmx.com wrote:
> From: Junyan He <junyan.he@intel.com>
> 
> QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration.
> If the backend is on the persistent memory, QEMU needs to take proper operations to
> ensure its writes persistent on the persistent memory. Otherwise, a host power failure
> may result in the loss the guest data on the persistent memory.

Ping Michael.  Can this go through your tree?

> 
> This patch series is based on Marcel's patch "mem: add share parameter to memory-backend-ram" [1]
> because of the changes in patch 1.
> [1] https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg03858.html
> 
> Previous versions of this patch series can be found at: 
> v10: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg03433.html
> v9: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg02361.html
> v8: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg02279.html
> v7: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg02997.html
> v6: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg00061.html
> v5: https://lists.gnu.org/archive/html/qemu-devel/2018-05/msg02258.html
> V4: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg06993.html
> v3: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg04365.html
> v2: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg01579.html
> v1: https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg05040.html
> 
> Changes in v11:
> * (Patch 2) Modify the ram_flags parameter to 32bits, the same size as it in RAMBlock
> * (Patch 5 and Patch 7) Delete pmem_xxx stub functions in stubs/pmem.c. Use inline
> functions with assert to replace them, because we never come there when pmem is enabled
> but lack of libpmem support.
> 
> Changes in v10:
> * (Patch 4) Fix a nit in nvdimm docs about pmem option usage in command line 
> The v10 patch set is all reviewed by Igor Mammedov <imammedo@redhat.com>
> 
> Changes in v9:
> * (Patch 3 and Patch 4) Reorder these two patches to make logic right.
> Firstly add libpmem support, and then we can use libpmem's configure
> check result. Also fix some typo and grammar issues in these two patches.
> 
> Changs in v8: 
> * (Patch 3) Report a error when user set 'pmem' to file-backend, while
> the qemu is lack of libpmem support. In this case, we can not ensure
> the persistence of the file-backend, so we choose to fail the build
> rather than contine and make the thing more confused.
> 
> Changes in v7: 
> The v6 patch set has already reviewed by Stefan Hajnoczi <stefanha@redhat.com>
> No logic change in this v7 version, just:
> * Spelling check and some document words refined.
> * Rebase to "ram is migratable" patch set.
> 
> Changes in v6: 
> * (Patch 1) Expose all ram block flags rather than redefine the flags.
> * (Patch 4) Use pkg-config rather the hard check when configure. 
> * (Patch 7) Sync and flush all the pmem data when migration completes,
> rather than sync pages one by one in previous version.
> 
> Changes in v5: 
> * (Patch 9) Add post copy check and output some messages for nvdimm.
> 
> Changes in v4: 
> * (Patch 2) Fix compilation errors found by patchew.
> 
> Changes in v3:
> * (Patch 5) Add a is_pmem flag to ram_handle_compressed() and handle
> PMEM writes in it, so we don't need the _common function.
> * (Patch 6) Expose qemu_get_buffer_common so we can remove the
> unnecessary qemu_get_buffer_to_pmem wrapper.
> * (Patch 8) Add a is_pmem flag to xbzrle_decode_buffer() and handle
> PMEM writes in it, so we can remove the unnecessary
> xbzrle_decode_buffer_{common, to_pmem}.
> * Move libpmem stubs to stubs/pmem.c and fix the compilation failures
> of test-{xbzrle,vmstate}.c.
> 
> Changes in v2:
> * (Patch 1) Use a flags parameter in file ram allocation functions.
> * (Patch 2) Add a new option 'pmem' to hostmem-file.
> * (Patch 3) Use libpmem to operate on the persistent memory, rather
> than re-implementing those operations in QEMU.
> * (Patch 5-8) Consider the write persistence in the migration path.
> 
> 
> Junyan:
> [1/7] memory, exec: Expose all memory block related flags.
> [6/7] migration/ram: Add check and info message to nvdimm post copy.
> [7/7] migration/ram: ensure write persistence on loading all date to PMEM.
> 
> Haozhong:
> [5/7] mem/nvdimm: ensure write persistence to PMEM in label emulation
> 
> Haozhong & Junyan:
> [2/7] memory, exec: switch file ram allocation functions to 'flags' parameters
> [3/7] configure: add libpmem support [4/7] hostmem-file: add the 'pmem' option
> [4/7] hostmem-file: add the 'pmem' option
> 
> --
> backends/hostmem-file.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
> configure               | 29 +++++++++++++++++++++++++++++
> docs/nvdimm.txt         | 22 ++++++++++++++++++++++
> exec.c                  | 38 +++++++++++++-------------------------
> hw/mem/nvdimm.c         |  9 ++++++++-
> include/exec/memory.h   | 31 +++++++++++++++++++++++++++++--
> include/exec/ram_addr.h | 28 ++++++++++++++++++++++++++--
> include/qemu/pmem.h     | 36 ++++++++++++++++++++++++++++++++++++
> memory.c                |  8 +++++---
> migration/ram.c         | 17 +++++++++++++++++
> numa.c                  |  2 +-
> qemu-options.hx         |  7 +++++++
> 12 files changed, 235 insertions(+), 36 deletions(-)
> create mode 100644 include/qemu/pmem.h
> 
> -- 
> 2.7.4
> 
> 
Re: [Qemu-devel] [PATCH 0/7 V11] nvdimm: guarantee persistence of QEMU writes to persistent memory
Posted by Michael S. Tsirkin 5 years, 8 months ago
On Fri, Jul 27, 2018 at 01:49:17PM +0100, Stefan Hajnoczi wrote:
> On Wed, Jul 18, 2018 at 03:47:56PM +0800, junyan.he@gmx.com wrote:
> > From: Junyan He <junyan.he@intel.com>
> > 
> > QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration.
> > If the backend is on the persistent memory, QEMU needs to take proper operations to
> > ensure its writes persistent on the persistent memory. Otherwise, a host power failure
> > may result in the loss the guest data on the persistent memory.
> 
> Ping Michael.  Can this go through your tree?

OK but do you really think it's appropriate in this release cycle?
If not pls repost after the release including the acks.

> > 
> > This patch series is based on Marcel's patch "mem: add share parameter to memory-backend-ram" [1]
> > because of the changes in patch 1.
> > [1] https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg03858.html
> > 
> > Previous versions of this patch series can be found at: 
> > v10: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg03433.html
> > v9: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg02361.html
> > v8: https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg02279.html
> > v7: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg02997.html
> > v6: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg00061.html
> > v5: https://lists.gnu.org/archive/html/qemu-devel/2018-05/msg02258.html
> > V4: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg06993.html
> > v3: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg04365.html
> > v2: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg01579.html
> > v1: https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg05040.html
> > 
> > Changes in v11:
> > * (Patch 2) Modify the ram_flags parameter to 32bits, the same size as it in RAMBlock
> > * (Patch 5 and Patch 7) Delete pmem_xxx stub functions in stubs/pmem.c. Use inline
> > functions with assert to replace them, because we never come there when pmem is enabled
> > but lack of libpmem support.
> > 
> > Changes in v10:
> > * (Patch 4) Fix a nit in nvdimm docs about pmem option usage in command line 
> > The v10 patch set is all reviewed by Igor Mammedov <imammedo@redhat.com>
> > 
> > Changes in v9:
> > * (Patch 3 and Patch 4) Reorder these two patches to make logic right.
> > Firstly add libpmem support, and then we can use libpmem's configure
> > check result. Also fix some typo and grammar issues in these two patches.
> > 
> > Changs in v8: 
> > * (Patch 3) Report a error when user set 'pmem' to file-backend, while
> > the qemu is lack of libpmem support. In this case, we can not ensure
> > the persistence of the file-backend, so we choose to fail the build
> > rather than contine and make the thing more confused.
> > 
> > Changes in v7: 
> > The v6 patch set has already reviewed by Stefan Hajnoczi <stefanha@redhat.com>
> > No logic change in this v7 version, just:
> > * Spelling check and some document words refined.
> > * Rebase to "ram is migratable" patch set.
> > 
> > Changes in v6: 
> > * (Patch 1) Expose all ram block flags rather than redefine the flags.
> > * (Patch 4) Use pkg-config rather the hard check when configure. 
> > * (Patch 7) Sync and flush all the pmem data when migration completes,
> > rather than sync pages one by one in previous version.
> > 
> > Changes in v5: 
> > * (Patch 9) Add post copy check and output some messages for nvdimm.
> > 
> > Changes in v4: 
> > * (Patch 2) Fix compilation errors found by patchew.
> > 
> > Changes in v3:
> > * (Patch 5) Add a is_pmem flag to ram_handle_compressed() and handle
> > PMEM writes in it, so we don't need the _common function.
> > * (Patch 6) Expose qemu_get_buffer_common so we can remove the
> > unnecessary qemu_get_buffer_to_pmem wrapper.
> > * (Patch 8) Add a is_pmem flag to xbzrle_decode_buffer() and handle
> > PMEM writes in it, so we can remove the unnecessary
> > xbzrle_decode_buffer_{common, to_pmem}.
> > * Move libpmem stubs to stubs/pmem.c and fix the compilation failures
> > of test-{xbzrle,vmstate}.c.
> > 
> > Changes in v2:
> > * (Patch 1) Use a flags parameter in file ram allocation functions.
> > * (Patch 2) Add a new option 'pmem' to hostmem-file.
> > * (Patch 3) Use libpmem to operate on the persistent memory, rather
> > than re-implementing those operations in QEMU.
> > * (Patch 5-8) Consider the write persistence in the migration path.
> > 
> > 
> > Junyan:
> > [1/7] memory, exec: Expose all memory block related flags.
> > [6/7] migration/ram: Add check and info message to nvdimm post copy.
> > [7/7] migration/ram: ensure write persistence on loading all date to PMEM.
> > 
> > Haozhong:
> > [5/7] mem/nvdimm: ensure write persistence to PMEM in label emulation
> > 
> > Haozhong & Junyan:
> > [2/7] memory, exec: switch file ram allocation functions to 'flags' parameters
> > [3/7] configure: add libpmem support [4/7] hostmem-file: add the 'pmem' option
> > [4/7] hostmem-file: add the 'pmem' option
> > 
> > --
> > backends/hostmem-file.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
> > configure               | 29 +++++++++++++++++++++++++++++
> > docs/nvdimm.txt         | 22 ++++++++++++++++++++++
> > exec.c                  | 38 +++++++++++++-------------------------
> > hw/mem/nvdimm.c         |  9 ++++++++-
> > include/exec/memory.h   | 31 +++++++++++++++++++++++++++++--
> > include/exec/ram_addr.h | 28 ++++++++++++++++++++++++++--
> > include/qemu/pmem.h     | 36 ++++++++++++++++++++++++++++++++++++
> > memory.c                |  8 +++++---
> > migration/ram.c         | 17 +++++++++++++++++
> > numa.c                  |  2 +-
> > qemu-options.hx         |  7 +++++++
> > 12 files changed, 235 insertions(+), 36 deletions(-)
> > create mode 100644 include/qemu/pmem.h
> > 
> > -- 
> > 2.7.4
> > 
> > 



Re: [Qemu-devel] [PATCH 0/7 V11] nvdimm: guarantee persistence of QEMU writes to persistent memory
Posted by Stefan Hajnoczi 5 years, 8 months ago
On Fri, Jul 27, 2018 at 04:06:23PM +0300, Michael S. Tsirkin wrote:
> On Fri, Jul 27, 2018 at 01:49:17PM +0100, Stefan Hajnoczi wrote:
> > On Wed, Jul 18, 2018 at 03:47:56PM +0800, junyan.he@gmx.com wrote:
> > > From: Junyan He <junyan.he@intel.com>
> > > 
> > > QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration.
> > > If the backend is on the persistent memory, QEMU needs to take proper operations to
> > > ensure its writes persistent on the persistent memory. Otherwise, a host power failure
> > > may result in the loss the guest data on the persistent memory.
> > 
> > Ping Michael.  Can this go through your tree?
> 
> OK but do you really think it's appropriate in this release cycle?
> If not pls repost after the release including the acks.

I agree, it's too late for QEMU 3.0.

Have you considered keeping a -next tree?  It's more
contributor-friendly if you merge patches into a -next tree so they
don't need to worry about it anymore.

Stefan