Hi all!

It's an alternative lock-less solution to
  [PATCH v4 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

In v6 a lot of things are rewritten.

What is changed:

1. rename the feature to host_range_refcnt, move it to separate file
2. better naming for everything (I hope)
3. cover reads, not only writes
4. do "ref" in qcow2_get_host_offset(), qcow2_alloc_host_offset(),
    qcow2_alloc_compressed_cluster_offset().
   and callers do "unref" appropriately.

Vladimir Sementsov-Ogievskiy (12):
  iotests: add qcow2-discard-during-rewrite
  qcow2: fix cache discarding in update_refcount()
  block/qcow2-cluster: assert no data_file on compressed write path
  block/qcow2-refcount: rename and publish update_refcount_discard()
  block/qcow2: introduce qcow2_parse_compressed_cluster_descriptor()
  block/qcow2: refactor qcow2_co_preadv_task() to have one return
  block/qcow2: qcow2_co_pwrite_zeroes: use QEMU_LOCK_GUARD
  qcow2: introduce is_cluster_free() helper
  qcow2: introduce host-range-refs
  qcow2: introduce qcow2_host_cluster_postponed_discard()
  qcow2: protect data writing by host range reference
  qcow2: protect data reading by host range reference

 block/qcow2.h                                 |  26 +++
 block/qcow2-cluster.c                         |  55 +++++-
 block/qcow2-host-range-refs.c                 | 174 ++++++++++++++++++
 block/qcow2-refcount.c                        |  61 ++++--
 block/qcow2.c                                 | 118 ++++++++----
 block/meson.build                             |   1 +
 .../tests/qcow2-discard-during-rewrite        |  72 ++++++++
 .../tests/qcow2-discard-during-rewrite.out    |  21 +++
 8 files changed, 475 insertions(+), 53 deletions(-)
 create mode 100644 block/qcow2-host-range-refs.c
 create mode 100755 tests/qemu-iotests/tests/qcow2-discard-during-rewrite
 create mode 100644 tests/qemu-iotests/tests/qcow2-discard-during-rewrite.out

-- 
2.29.2

22.04.2021 19:30, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> It's an alternative lock-less solution to
>    [PATCH v4 0/3] qcow2: fix parallel rewrite and discard (rw-lock)
> 
> In v6 a lot of things are rewritten.
> 
> What is changed:
> 
> 1. rename the feature to host_range_refcnt, move it to separate file
> 2. better naming for everything (I hope)
> 3. cover reads, not only writes
> 4. do "ref" in qcow2_get_host_offset(), qcow2_alloc_host_offset(),
>      qcow2_alloc_compressed_cluster_offset().
>     and callers do "unref" appropriately.
> 


About performance. With these series we do extra allocations and hash-map operations.. Still testing by

./build/qemu-img bench -c 1000000 -s 4K --image-opts driver=null-co,size=5G

and

./build/qemu-img bench -c 1000000 -s 4K -w --image-opts driver=null-co,size=5G

I see difference less than 1%.


-- 
Best regards,
Vladimir

Hi!

Kevin, what do think on this now? Do you think "[PATCH v4 0/3] qcow2: fix parallel rewrite and discard (rw-lock)" still worth to update to cover reads and resend? (consider also my replies to "[PATCH v4 0/3] qcow2: fix parallel rewrite and discard (rw-lock)" branch)

22.04.2021 19:30, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> It's an alternative lock-less solution to
>    [PATCH v4 0/3] qcow2: fix parallel rewrite and discard (rw-lock)
> 
> In v6 a lot of things are rewritten.
> 
> What is changed:
> 
> 1. rename the feature to host_range_refcnt, move it to separate file
> 2. better naming for everything (I hope)
> 3. cover reads, not only writes
> 4. do "ref" in qcow2_get_host_offset(), qcow2_alloc_host_offset(),
>      qcow2_alloc_compressed_cluster_offset().
>     and callers do "unref" appropriately.
> 
> Vladimir Sementsov-Ogievskiy (12):
>    iotests: add qcow2-discard-during-rewrite
>    qcow2: fix cache discarding in update_refcount()
>    block/qcow2-cluster: assert no data_file on compressed write path
>    block/qcow2-refcount: rename and publish update_refcount_discard()
>    block/qcow2: introduce qcow2_parse_compressed_cluster_descriptor()
>    block/qcow2: refactor qcow2_co_preadv_task() to have one return
>    block/qcow2: qcow2_co_pwrite_zeroes: use QEMU_LOCK_GUARD
>    qcow2: introduce is_cluster_free() helper
>    qcow2: introduce host-range-refs
>    qcow2: introduce qcow2_host_cluster_postponed_discard()
>    qcow2: protect data writing by host range reference
>    qcow2: protect data reading by host range reference
> 
>   block/qcow2.h                                 |  26 +++
>   block/qcow2-cluster.c                         |  55 +++++-
>   block/qcow2-host-range-refs.c                 | 174 ++++++++++++++++++
>   block/qcow2-refcount.c                        |  61 ++++--
>   block/qcow2.c                                 | 118 ++++++++----
>   block/meson.build                             |   1 +
>   .../tests/qcow2-discard-during-rewrite        |  72 ++++++++
>   .../tests/qcow2-discard-during-rewrite.out    |  21 +++
>   8 files changed, 475 insertions(+), 53 deletions(-)
>   create mode 100644 block/qcow2-host-range-refs.c
>   create mode 100755 tests/qemu-iotests/tests/qcow2-discard-during-rewrite
>   create mode 100644 tests/qemu-iotests/tests/qcow2-discard-during-rewrite.out
> 


-- 
Best regards,
Vladimir