The following changes since commit 8f72c75cfc9b3c84a9b5e7a58ee5e471cb2f19c8:

  Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200526-pull-request' into staging (2020-05-26 10:59:01 +0100)

are available in the Git repository at:

  https://repo.or.cz/qemu/ericb.git tags/pull-bitmaps-2020-05-26

for you to fetch changes up to 9f2bcd08bce234f67239aaf6f4b881ccf0a76d55:

  iotests: Add test 291 to for qemu-img bitmap coverage (2020-05-26 11:28:24 -0500)

----------------------------------------------------------------
bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'

----------------------------------------------------------------
Eric Blake (5):
      iotests: Fix test 178
      qcow2: Expose bitmaps' size during measure
      qemu-img: Factor out code for merging bitmaps
      qemu-img: Add convert --bitmaps option
      iotests: Add test 291 to for qemu-img bitmap coverage

Vladimir Sementsov-Ogievskiy (6):
      migration: refactor init_dirty_bitmap_migration
      block/dirty-bitmap: add bdrv_has_named_bitmaps helper
      migration: fix bitmaps pre-blockdev migration with mirror job
      iotests: 194: test also migration of dirty bitmap
      migration: add_bitmaps_to_list: check disk name once
      migration: forbid bitmap migration by generated node-name

 docs/tools/qemu-img.rst          |  13 +++-
 qapi/block-core.json             |  16 +++--
 block/qcow2.h                    |   2 +
 include/block/dirty-bitmap.h     |   1 +
 block/crypto.c                   |   2 +-
 block/dirty-bitmap.c             |  13 ++++
 block/qcow2-bitmap.c             |  36 ++++++++++
 block/qcow2.c                    |  14 +++-
 block/raw-format.c               |   2 +-
 migration/block-dirty-bitmap.c   | 142 ++++++++++++++++++++++++++++-----------
 qemu-img.c                       | 107 ++++++++++++++++++++++++-----
 qemu-img-cmds.hx                 |   4 +-
 tests/qemu-iotests/178.out.qcow2 |  18 ++++-
 tests/qemu-iotests/178.out.raw   |   2 +-
 tests/qemu-iotests/190           |  45 ++++++++++++-
 tests/qemu-iotests/190.out       |  27 +++++++-
 tests/qemu-iotests/194           |  14 ++--
 tests/qemu-iotests/194.out       |   6 ++
 tests/qemu-iotests/291           | 112 ++++++++++++++++++++++++++++++
 tests/qemu-iotests/291.out       |  80 ++++++++++++++++++++++
 tests/qemu-iotests/group         |   1 +
 21 files changed, 580 insertions(+), 77 deletions(-)
 create mode 100755 tests/qemu-iotests/291
 create mode 100644 tests/qemu-iotests/291.out

-- 
2.26.2

On Tue, 26 May 2020 at 17:43, Eric Blake <eblake@redhat.com> wrote:
>
> The following changes since commit 8f72c75cfc9b3c84a9b5e7a58ee5e471cb2f19c8:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200526-pull-request' into staging (2020-05-26 10:59:01 +0100)
>
> are available in the Git repository at:
>
>   https://repo.or.cz/qemu/ericb.git tags/pull-bitmaps-2020-05-26
>
> for you to fetch changes up to 9f2bcd08bce234f67239aaf6f4b881ccf0a76d55:
>
>   iotests: Add test 291 to for qemu-img bitmap coverage (2020-05-26 11:28:24 -0500)
>
> ----------------------------------------------------------------
> bitmaps patches for 2020-05-26
>
> - fix non-blockdev migration of bitmaps when mirror job is in use
> - add bitmap sizing to 'qemu-img measure'
> - add 'qemu-img convert --bitmaps'
>
> ----------------------------------------------------------------


iotest 190 failed on freebsd:

  TEST    iotest-qcow2: 190 [fail]
QEMU          --
"/home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodefaults -d
isplay none -machine virt -accel qtest
QEMU_IMG      --
"/home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2
QEMU_NBD      --
"/home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- NetBSD/amd64 localhost 9.0
TEST_DIR      -- /home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/mktemp.MJWbGXjT
SOCKET_SCM_HELPER --

--- /home/qemu/qemu-test.BE3Bvf/src/tests/qemu-iotests/190.out
2020-05-27 15:13:27.000000000 +0000
+++ /home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/190.out.bad
 2020-05-27 15:30:50.377759533 +0000
@@ -17,7 +17,7 @@
 fully allocated size: 10813440
 required size: 2199023255552
 fully allocated size: 2199023255552
-required size: 7012352
+required size: 17170432
 fully allocated size: 17170432
 required size: 335806464
 fully allocated size: 2199359062016

NetBSD failed too but I'm not sure if it was the same thing
or just that it's being very unreliable currently.

thanks
-- PMM

On 5/27/20 3:07 PM, Peter Maydell wrote:

> 
> iotest 190 failed on freebsd:
> 

> +++ /home/qemu/qemu-test.BE3Bvf/build/tests/qemu-iotests/190.out.bad
>   2020-05-27 15:30:50.377759533 +0000
> @@ -17,7 +17,7 @@
>   fully allocated size: 10813440
>   required size: 2199023255552
>   fully allocated size: 2199023255552
> -required size: 7012352
> +required size: 17170432
>   fully allocated size: 17170432

Thanks for the heads up.  That was on:

# No bitmap output, since no bitmaps on raw source
$QEMU_IMG measure -O qcow2 -f raw "$TEST_IMG"

and indicates that on FreeBSD, the qcow2 image is not as sparse as it is 
on other platforms.  Where Linux was able to punch holes in the 
underlying filesystem, FreeBSD did not.  But even though I'm not sure if 
that is due to file system hole granularity, choice of APIs used to 
write all-zero bitmaps, or something else, I am certain that it less 
important (the qcow2 file is still quite sparse in comparison to the 2T 
guest-visible data it is representing, even if it differs in sparseness 
between the systems).

I'll post a v2 that filters out the required size for just that command, 
as viewing a qcow2 file as if raw is unusual.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org