[PATCH v4 0/5] block: add block-dirty-bitmap-populate job

Eric Blake posted 5 patches 3 years, 8 months ago
Test docker-quick@centos7 failed
Test docker-mingw@fedora failed
Test checkpatch failed
Test FreeBSD failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200902181831.2570048-1-eblake@redhat.com
Maintainers: Max Reitz <mreitz@redhat.com>, Eric Blake <eblake@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Markus Armbruster <armbru@redhat.com>, John Snow <jsnow@redhat.com>
qapi/block-core.json          |   66 +
qapi/job.json                 |    6 +-
qapi/transaction.json         |    2 +
include/block/block.h         |    1 +
include/block/block_int.h     |   21 +
block/bitmap-populate.c       |  207 ++
blockdev.c                    |  104 +-
blockjob.c                    |    3 +-
MAINTAINERS                   |    1 +
block/meson.build             |    1 +
tests/qemu-iotests/257        |  110 +-
tests/qemu-iotests/298        |  232 ++
tests/qemu-iotests/298.out    | 4544 +++++++++++++++++++++++++++++++++
tests/qemu-iotests/bitmaps.py |  131 +
tests/qemu-iotests/group      |    1 +
15 files changed, 5301 insertions(+), 129 deletions(-)
create mode 100644 block/bitmap-populate.c
create mode 100755 tests/qemu-iotests/298
create mode 100644 tests/qemu-iotests/298.out
create mode 100644 tests/qemu-iotests/bitmaps.py
[PATCH v4 0/5] block: add block-dirty-bitmap-populate job
Posted by Eric Blake 3 years, 8 months ago
This is NOT the final version of this patch series, but I'm posting it
to revive conversation on the topic while fixing it to compile on top
of meson changes.

v3 was: https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06258.html

001/5:[0025] [FC] 'block: add bitmap-populate job'
002/5:[----] [--] 'blockdev: combine DriveBackupState and BlockdevBackupState'
003/5:[0004] [FC] 'qmp: expose block-dirty-bitmap-populate'
004/5:[----] [--] 'iotests: move bitmap helpers into their own file'
005/5:[----] [-C] 'iotests: add 298 for block-dirty-bitmap-populate'

I'm still trying to find the right QAPI contract (affects patch 1 and
3 for the qapi, and 5 for invoking the command in iotests), but right
now, I'm leaning towards:

{ "execute": "block-dirty-bitmap-populate", "arguments": {
    "job-id": "job0", "node": "target_node", "name": "target_bitmap",
    "sources": [ { "pattern": "allocation", "node": "from_node" } ] } }

which allows expansion into multiple sources, so that we can combine a
populate action with a bitmap merge rather than having to do those as
separate commands, like:

{ "execute": "block-dirty-bitmap-populate", "arguments": {
    "job-id": "job0", "node": "target_node", "name": "target_bitmap",
    "sources": [ { "pattern": "allocation", "node": "from_node" },
                 { "pattern": "bitmap", "node": "from_node",
		     "bitmap": "from_bitmap" } ] } }

John Snow (5):
  block: add bitmap-populate job
  blockdev: combine DriveBackupState and BlockdevBackupState
  qmp: expose block-dirty-bitmap-populate
  iotests: move bitmap helpers into their own file
  iotests: add 298 for block-dirty-bitmap-populate

 qapi/block-core.json          |   66 +
 qapi/job.json                 |    6 +-
 qapi/transaction.json         |    2 +
 include/block/block.h         |    1 +
 include/block/block_int.h     |   21 +
 block/bitmap-populate.c       |  207 ++
 blockdev.c                    |  104 +-
 blockjob.c                    |    3 +-
 MAINTAINERS                   |    1 +
 block/meson.build             |    1 +
 tests/qemu-iotests/257        |  110 +-
 tests/qemu-iotests/298        |  232 ++
 tests/qemu-iotests/298.out    | 4544 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/bitmaps.py |  131 +
 tests/qemu-iotests/group      |    1 +
 15 files changed, 5301 insertions(+), 129 deletions(-)
 create mode 100644 block/bitmap-populate.c
 create mode 100755 tests/qemu-iotests/298
 create mode 100644 tests/qemu-iotests/298.out
 create mode 100644 tests/qemu-iotests/bitmaps.py

-- 
2.28.0


Re: [PATCH v4 0/5] block: add block-dirty-bitmap-populate job
Posted by Vladimir Sementsov-Ogievskiy 3 years, 8 months ago
02.09.2020 21:18, Eric Blake wrote:
> This is NOT the final version of this patch series, but I'm posting it
> to revive conversation on the topic while fixing it to compile on top
> of meson changes.
> 
> v3 was: https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06258.html
> 
> 001/5:[0025] [FC] 'block: add bitmap-populate job'
> 002/5:[----] [--] 'blockdev: combine DriveBackupState and BlockdevBackupState'
> 003/5:[0004] [FC] 'qmp: expose block-dirty-bitmap-populate'
> 004/5:[----] [--] 'iotests: move bitmap helpers into their own file'
> 005/5:[----] [-C] 'iotests: add 298 for block-dirty-bitmap-populate'
> 
> I'm still trying to find the right QAPI contract (affects patch 1 and
> 3 for the qapi, and 5 for invoking the command in iotests), but right
> now, I'm leaning towards:
> 
> { "execute": "block-dirty-bitmap-populate", "arguments": {
>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",
>      "sources": [ { "pattern": "allocation", "node": "from_node" } ] } }
> 
> which allows expansion into multiple sources, so that we can combine a
> populate action with a bitmap merge rather than having to do those as
> separate commands, like:
> 
> { "execute": "block-dirty-bitmap-populate", "arguments": {
>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",
>      "sources": [ { "pattern": "allocation", "node": "from_node" },
>                   { "pattern": "bitmap", "node": "from_node",
> 		     "bitmap": "from_bitmap" } ] } }

(I see, this is not realized in v4)

What about multiple targets as well, as we discussed in v3?

> 
> John Snow (5):
>    block: add bitmap-populate job
>    blockdev: combine DriveBackupState and BlockdevBackupState
>    qmp: expose block-dirty-bitmap-populate
>    iotests: move bitmap helpers into their own file
>    iotests: add 298 for block-dirty-bitmap-populate
> 
>   qapi/block-core.json          |   66 +
>   qapi/job.json                 |    6 +-
>   qapi/transaction.json         |    2 +
>   include/block/block.h         |    1 +
>   include/block/block_int.h     |   21 +
>   block/bitmap-populate.c       |  207 ++
>   blockdev.c                    |  104 +-
>   blockjob.c                    |    3 +-
>   MAINTAINERS                   |    1 +
>   block/meson.build             |    1 +
>   tests/qemu-iotests/257        |  110 +-
>   tests/qemu-iotests/298        |  232 ++
>   tests/qemu-iotests/298.out    | 4544 +++++++++++++++++++++++++++++++++
>   tests/qemu-iotests/bitmaps.py |  131 +
>   tests/qemu-iotests/group      |    1 +
>   15 files changed, 5301 insertions(+), 129 deletions(-)
>   create mode 100644 block/bitmap-populate.c
>   create mode 100755 tests/qemu-iotests/298
>   create mode 100644 tests/qemu-iotests/298.out
>   create mode 100644 tests/qemu-iotests/bitmaps.py
> 


-- 
Best regards,
Vladimir

Re: [PATCH v4 0/5] block: add block-dirty-bitmap-populate job
Posted by Vladimir Sementsov-Ogievskiy 3 years, 1 month ago
Hi!

Any plans on v5? Now I have a request for this feature.. If no plans, I can take care of it and make a v5

02.09.2020 21:18, Eric Blake wrote:
> This is NOT the final version of this patch series, but I'm posting it
> to revive conversation on the topic while fixing it to compile on top
> of meson changes.
> 
> v3 was:https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06258.html
> 
> 001/5:[0025] [FC] 'block: add bitmap-populate job'
> 002/5:[----] [--] 'blockdev: combine DriveBackupState and BlockdevBackupState'
> 003/5:[0004] [FC] 'qmp: expose block-dirty-bitmap-populate'
> 004/5:[----] [--] 'iotests: move bitmap helpers into their own file'
> 005/5:[----] [-C] 'iotests: add 298 for block-dirty-bitmap-populate'
> 
> I'm still trying to find the right QAPI contract (affects patch 1 and
> 3 for the qapi, and 5 for invoking the command in iotests), but right
> now, I'm leaning towards:
> 
> { "execute": "block-dirty-bitmap-populate", "arguments": {
>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",
>      "sources": [ { "pattern": "allocation", "node": "from_node" } ] } }
> 
> which allows expansion into multiple sources, so that we can combine a
> populate action with a bitmap merge rather than having to do those as
> separate commands, like:
> 
> { "execute": "block-dirty-bitmap-populate", "arguments": {
>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",
>      "sources": [ { "pattern": "allocation", "node": "from_node" },
>                   { "pattern": "bitmap", "node": "from_node",
> 		     "bitmap": "from_bitmap" } ] } }


-- 
Best regards,
Vladimir