[Qemu-devel] [PATCH v3 00/11] make dirty-bitmap byte-based

Eric Blake posted 11 patches 6 years, 9 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
include/block/block_int.h    |   2 +-
include/block/dirty-bitmap.h |  28 ++++--------
block/backup.c               |   7 ++-
block/dirty-bitmap.c         | 105 +++++++++++--------------------------------
block/io.c                   |   6 +--
block/mirror.c               |  73 +++++++++++++-----------------
migration/block.c            |  12 +++--
7 files changed, 79 insertions(+), 154 deletions(-)
[Qemu-devel] [PATCH v3 00/11] make dirty-bitmap byte-based
Posted by Eric Blake 6 years, 9 months ago
There are patches floating around to add NBD_CMD_BLOCK_STATUS,
but NBD wants to report status on byte granularity (even if the
reporting will probably be naturally aligned to sectors or even
much higher levels).  I've therefore started the task of
converting our block status code to report at a byte granularity
rather than sectors.

This is part two of that conversion: dirty-bitmap. Other parts
include bdrv_is_allocated (at v3 [1]) and replacing
bdrv_get_block_status with a byte based callback in all the
drivers (at v1, needs a rebase [3]).

Available as a tag at:
git fetch git://repo.or.cz/qemu/ericb.git nbd-byte-dirty-v3

Depends on Kevin's block branch and my v3 bdrv_is_allocated [1]

Since v2 [2], I had to rebase on top of Paolo's locking fixes;
patch v2 2/12 is gone, and many of the others had a lot of
context conflicts. But I felt the resolution was simple enough
that I kept R-b on all but patch 8.

[1] https://lists.gnu.org/archive/html/qemu-devel/2017-06/msg06077.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg03859.html
[3] https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg02642.html

(git backport-diff doesn't like the rename in 7/11)

001/11:[----] [--] 'dirty-bitmap: Report BlockDirtyInfo.count in bytes, as documented'
002/11:[0024] [FC] 'dirty-bitmap: Drop unused functions'
003/11:[----] [-C] 'dirty-bitmap: Track size in bytes'
004/11:[----] [-C] 'dirty-bitmap: Set iterator start by offset, not sector'
005/11:[----] [-C] 'dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset'
006/11:[----] [-C] 'dirty-bitmap: Change bdrv_get_dirty_count() to report bytes'
007/11:[down] 'dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes'
008/11:[0036] [FC] 'dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes'
009/11:[----] [--] 'mirror: Switch mirror_dirty_init() to byte-based iteration'
010/11:[0001] [FC] 'dirty-bitmap: Switch bdrv_set_dirty() to bytes'
011/11:[----] [-C] 'dirty-bitmap: Convert internal hbitmap size/granularity'

Eric Blake (11):
  dirty-bitmap: Report BlockDirtyInfo.count in bytes, as documented
  dirty-bitmap: Drop unused functions
  dirty-bitmap: Track size in bytes
  dirty-bitmap: Set iterator start by offset, not sector
  dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset
  dirty-bitmap: Change bdrv_get_dirty_count() to report bytes
  dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes
  dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes
  mirror: Switch mirror_dirty_init() to byte-based iteration
  dirty-bitmap: Switch bdrv_set_dirty() to bytes
  dirty-bitmap: Convert internal hbitmap size/granularity

 include/block/block_int.h    |   2 +-
 include/block/dirty-bitmap.h |  28 ++++--------
 block/backup.c               |   7 ++-
 block/dirty-bitmap.c         | 105 +++++++++++--------------------------------
 block/io.c                   |   6 +--
 block/mirror.c               |  73 +++++++++++++-----------------
 migration/block.c            |  12 +++--
 7 files changed, 79 insertions(+), 154 deletions(-)

-- 
2.9.4


Re: [Qemu-devel] [PATCH v3 00/11] make dirty-bitmap byte-based
Posted by Vladimir Sementsov-Ogievskiy 6 years, 9 months ago
Hi!

Can we apply my "[PATCH v22 00/30] qcow2: persistent dirty bitmaps" 
first? It was already near to the victory a week ago, but I had to 
rebase it on new Paolo's patches.

28.06.2017 20:55, Eric Blake wrote:
> There are patches floating around to add NBD_CMD_BLOCK_STATUS,
> but NBD wants to report status on byte granularity (even if the
> reporting will probably be naturally aligned to sectors or even
> much higher levels).  I've therefore started the task of
> converting our block status code to report at a byte granularity
> rather than sectors.
>
> This is part two of that conversion: dirty-bitmap. Other parts
> include bdrv_is_allocated (at v3 [1]) and replacing
> bdrv_get_block_status with a byte based callback in all the
> drivers (at v1, needs a rebase [3]).
>
> Available as a tag at:
> git fetch git://repo.or.cz/qemu/ericb.git nbd-byte-dirty-v3
>
> Depends on Kevin's block branch and my v3 bdrv_is_allocated [1]
>
> Since v2 [2], I had to rebase on top of Paolo's locking fixes;
> patch v2 2/12 is gone, and many of the others had a lot of
> context conflicts. But I felt the resolution was simple enough
> that I kept R-b on all but patch 8.
>
> [1] https://lists.gnu.org/archive/html/qemu-devel/2017-06/msg06077.html
> [2] https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg03859.html
> [3] https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg02642.html
>
> (git backport-diff doesn't like the rename in 7/11)
>
> 001/11:[----] [--] 'dirty-bitmap: Report BlockDirtyInfo.count in bytes, as documented'
> 002/11:[0024] [FC] 'dirty-bitmap: Drop unused functions'
> 003/11:[----] [-C] 'dirty-bitmap: Track size in bytes'
> 004/11:[----] [-C] 'dirty-bitmap: Set iterator start by offset, not sector'
> 005/11:[----] [-C] 'dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset'
> 006/11:[----] [-C] 'dirty-bitmap: Change bdrv_get_dirty_count() to report bytes'
> 007/11:[down] 'dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes'
> 008/11:[0036] [FC] 'dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes'
> 009/11:[----] [--] 'mirror: Switch mirror_dirty_init() to byte-based iteration'
> 010/11:[0001] [FC] 'dirty-bitmap: Switch bdrv_set_dirty() to bytes'
> 011/11:[----] [-C] 'dirty-bitmap: Convert internal hbitmap size/granularity'
>
> Eric Blake (11):
>    dirty-bitmap: Report BlockDirtyInfo.count in bytes, as documented
>    dirty-bitmap: Drop unused functions
>    dirty-bitmap: Track size in bytes
>    dirty-bitmap: Set iterator start by offset, not sector
>    dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset
>    dirty-bitmap: Change bdrv_get_dirty_count() to report bytes
>    dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes
>    dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes
>    mirror: Switch mirror_dirty_init() to byte-based iteration
>    dirty-bitmap: Switch bdrv_set_dirty() to bytes
>    dirty-bitmap: Convert internal hbitmap size/granularity
>
>   include/block/block_int.h    |   2 +-
>   include/block/dirty-bitmap.h |  28 ++++--------
>   block/backup.c               |   7 ++-
>   block/dirty-bitmap.c         | 105 +++++++++++--------------------------------
>   block/io.c                   |   6 +--
>   block/mirror.c               |  73 +++++++++++++-----------------
>   migration/block.c            |  12 +++--
>   7 files changed, 79 insertions(+), 154 deletions(-)
>

-- 
Best regards,
Vladimir


Re: [Qemu-devel] [PATCH v3 00/11] make dirty-bitmap byte-based
Posted by Eric Blake 6 years, 9 months ago
On 06/29/2017 03:28 AM, Vladimir Sementsov-Ogievskiy wrote:
> Hi!
> 
> Can we apply my "[PATCH v22 00/30] qcow2: persistent dirty bitmaps"
> first? It was already near to the victory a week ago, but I had to
> rebase it on new Paolo's patches.

Sure, I can rebase on top of yours (I've already rebased multiple times,
as have you, but yours has been on the list longer).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v3 00/11] make dirty-bitmap byte-based
Posted by John Snow 6 years, 9 months ago

On 06/29/2017 11:11 AM, Eric Blake wrote:
> On 06/29/2017 03:28 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Hi!
>>
>> Can we apply my "[PATCH v22 00/30] qcow2: persistent dirty bitmaps"
>> first? It was already near to the victory a week ago, but I had to
>> rebase it on new Paolo's patches.
> 
> Sure, I can rebase on top of yours (I've already rebased multiple times,
> as have you, but yours has been on the list longer).
> 

I would normally like to feign impartiality, but in this case I'd really
prefer if Vladimir's patches went through first, too.

--js