[PATCH v2 00/19] Make image fleecing more usable

Vladimir Sementsov-Ogievskiy posted 19 patches 2 years, 8 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
qapi/block-core.json                        |  39 ++-
block/fleecing.h                            | 151 ++++++++++++
include/block/block-copy.h                  |   4 +-
include/block/block_int.h                   |   1 +
include/block/dirty-bitmap.h                |   4 +-
include/block/reqlist.h                     |  75 ++++++
include/qemu/hbitmap.h                      |  11 +
include/sysemu/block-backend.h              |   2 +-
block/backup.c                              |  61 ++++-
block/block-backend.c                       |   2 +-
block/block-copy.c                          | 157 +++++-------
block/copy-before-write.c                   |  70 +++++-
block/dirty-bitmap.c                        |  15 +-
block/fleecing-drv.c                        | 260 ++++++++++++++++++++
block/fleecing.c                            | 182 ++++++++++++++
block/monitor/bitmap-qmp-cmds.c             |   5 +-
block/replication.c                         |   2 +-
block/reqlist.c                             |  84 +++++++
blockdev.c                                  |   1 +
util/hbitmap.c                              |  36 +++
MAINTAINERS                                 |   7 +-
block/meson.build                           |   3 +
tests/qemu-iotests/iotests.py               |   4 +
tests/qemu-iotests/tests/image-fleecing     | 178 +++++++++++---
tests/qemu-iotests/tests/image-fleecing.out | 221 ++++++++++++++++-
25 files changed, 1420 insertions(+), 155 deletions(-)
create mode 100644 block/fleecing.h
create mode 100644 include/block/reqlist.h
create mode 100644 block/fleecing-drv.c
create mode 100644 block/fleecing.c
create mode 100644 block/reqlist.c
[PATCH v2 00/19] Make image fleecing more usable
Posted by Vladimir Sementsov-Ogievskiy 2 years, 8 months ago
Hi all!

That continues "[PATCH RFC DRAFT 00/11] Make image fleecing more usable"
and supersedes "[PATCH v2 for-6.2 0/6] push backup with fleecing"

Supersedes: <20210804131750.127574-1-vsementsov@virtuozzo.com>
Supersedes: <20210721140424.163701-1-vsementsov@virtuozzo.com>

There several improvements to fleecing scheme:

1. support bitmap in copy-before-write filter

2. introduce fleecing block driver, which opens the door for a lot of
   image fleecing improvements.
   See "block: introduce fleecing block driver" commit message for
   details.

3. support "push backup with fleecing" scheme, when backup job is a
   client of common fleecing scheme. That helps when writes to final
   backup target are slow and we don't want guest writes hang waiting
   for copy-before-write operations to final target.

Vladimir Sementsov-Ogievskiy (19):
  block/block-copy: move copy_bitmap initialization to
    block_copy_state_new()
  block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
  block/block-copy: block_copy_state_new(): add bitmap parameter
  block/copy-before-write: add bitmap open parameter
  block/block-copy: add block_copy_reset()
  block: intoduce reqlist
  block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
  block/reqlist: add reqlist_wait_all()
  block: introduce FleecingState class
  block: introduce fleecing block driver
  block/copy-before-write: support fleecing block driver
  block/block-copy: add write-unchanged mode
  block/copy-before-write: use write-unchanged in fleecing mode
  iotests/image-fleecing: add test-case for fleecing format node
  iotests.py: add qemu_io_pipe_and_status()
  iotests/image-fleecing: add test case with bitmap
  block: blk_root(): return non-const pointer
  qapi: backup: add immutable-source parameter
  iotests/image-fleecing: test push backup with fleecing

 qapi/block-core.json                        |  39 ++-
 block/fleecing.h                            | 151 ++++++++++++
 include/block/block-copy.h                  |   4 +-
 include/block/block_int.h                   |   1 +
 include/block/dirty-bitmap.h                |   4 +-
 include/block/reqlist.h                     |  75 ++++++
 include/qemu/hbitmap.h                      |  11 +
 include/sysemu/block-backend.h              |   2 +-
 block/backup.c                              |  61 ++++-
 block/block-backend.c                       |   2 +-
 block/block-copy.c                          | 157 +++++-------
 block/copy-before-write.c                   |  70 +++++-
 block/dirty-bitmap.c                        |  15 +-
 block/fleecing-drv.c                        | 260 ++++++++++++++++++++
 block/fleecing.c                            | 182 ++++++++++++++
 block/monitor/bitmap-qmp-cmds.c             |   5 +-
 block/replication.c                         |   2 +-
 block/reqlist.c                             |  84 +++++++
 blockdev.c                                  |   1 +
 util/hbitmap.c                              |  36 +++
 MAINTAINERS                                 |   7 +-
 block/meson.build                           |   3 +
 tests/qemu-iotests/iotests.py               |   4 +
 tests/qemu-iotests/tests/image-fleecing     | 178 +++++++++++---
 tests/qemu-iotests/tests/image-fleecing.out | 221 ++++++++++++++++-
 25 files changed, 1420 insertions(+), 155 deletions(-)
 create mode 100644 block/fleecing.h
 create mode 100644 include/block/reqlist.h
 create mode 100644 block/fleecing-drv.c
 create mode 100644 block/fleecing.c
 create mode 100644 block/reqlist.c

-- 
2.29.2


Re: [PATCH v2 00/19] Make image fleecing more usable
Posted by Vladimir Sementsov-Ogievskiy 2 years, 7 months ago
ping )

27.08.2021 21:17, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> That continues "[PATCH RFC DRAFT 00/11] Make image fleecing more usable"
> and supersedes "[PATCH v2 for-6.2 0/6] push backup with fleecing"
> 
> Supersedes: <20210804131750.127574-1-vsementsov@virtuozzo.com>
> Supersedes: <20210721140424.163701-1-vsementsov@virtuozzo.com>
> 
> There several improvements to fleecing scheme:
> 
> 1. support bitmap in copy-before-write filter
> 
> 2. introduce fleecing block driver, which opens the door for a lot of
>     image fleecing improvements.
>     See "block: introduce fleecing block driver" commit message for
>     details.
> 
> 3. support "push backup with fleecing" scheme, when backup job is a
>     client of common fleecing scheme. That helps when writes to final
>     backup target are slow and we don't want guest writes hang waiting
>     for copy-before-write operations to final target.
> 
> Vladimir Sementsov-Ogievskiy (19):
>    block/block-copy: move copy_bitmap initialization to
>      block_copy_state_new()
>    block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
>    block/block-copy: block_copy_state_new(): add bitmap parameter
>    block/copy-before-write: add bitmap open parameter
>    block/block-copy: add block_copy_reset()
>    block: intoduce reqlist
>    block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
>    block/reqlist: add reqlist_wait_all()
>    block: introduce FleecingState class
>    block: introduce fleecing block driver
>    block/copy-before-write: support fleecing block driver
>    block/block-copy: add write-unchanged mode
>    block/copy-before-write: use write-unchanged in fleecing mode
>    iotests/image-fleecing: add test-case for fleecing format node
>    iotests.py: add qemu_io_pipe_and_status()
>    iotests/image-fleecing: add test case with bitmap
>    block: blk_root(): return non-const pointer
>    qapi: backup: add immutable-source parameter
>    iotests/image-fleecing: test push backup with fleecing
> 
>   qapi/block-core.json                        |  39 ++-
>   block/fleecing.h                            | 151 ++++++++++++
>   include/block/block-copy.h                  |   4 +-
>   include/block/block_int.h                   |   1 +
>   include/block/dirty-bitmap.h                |   4 +-
>   include/block/reqlist.h                     |  75 ++++++
>   include/qemu/hbitmap.h                      |  11 +
>   include/sysemu/block-backend.h              |   2 +-
>   block/backup.c                              |  61 ++++-
>   block/block-backend.c                       |   2 +-
>   block/block-copy.c                          | 157 +++++-------
>   block/copy-before-write.c                   |  70 +++++-
>   block/dirty-bitmap.c                        |  15 +-
>   block/fleecing-drv.c                        | 260 ++++++++++++++++++++
>   block/fleecing.c                            | 182 ++++++++++++++
>   block/monitor/bitmap-qmp-cmds.c             |   5 +-
>   block/replication.c                         |   2 +-
>   block/reqlist.c                             |  84 +++++++
>   blockdev.c                                  |   1 +
>   util/hbitmap.c                              |  36 +++
>   MAINTAINERS                                 |   7 +-
>   block/meson.build                           |   3 +
>   tests/qemu-iotests/iotests.py               |   4 +
>   tests/qemu-iotests/tests/image-fleecing     | 178 +++++++++++---
>   tests/qemu-iotests/tests/image-fleecing.out | 221 ++++++++++++++++-
>   25 files changed, 1420 insertions(+), 155 deletions(-)
>   create mode 100644 block/fleecing.h
>   create mode 100644 include/block/reqlist.h
>   create mode 100644 block/fleecing-drv.c
>   create mode 100644 block/fleecing.c
>   create mode 100644 block/reqlist.c
> 


-- 
Best regards,
Vladimir