[PATCH 0/6] Make blockdev-mirror dest sparse in more cases

Eric Blake posted 6 patches 5 months, 3 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250411010732.358817-8-eblake@redhat.com
Maintainers: Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, John Snow <jsnow@redhat.com>, Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>, Stefan Hajnoczi <stefanha@redhat.com>, Fam Zheng <fam@euphon.net>, Ronnie Sahlberg <ronniesahlberg@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>, Peter Lieven <pl@dlhnet.de>, Eric Blake <eblake@redhat.com>, "Denis V. Lunev" <den@openvz.org>, Alberto Garcia <berto@igalia.com>, Ilya Dryomov <idryomov@gmail.com>, Stefan Weil <sw@weilnetz.de>, Markus Armbruster <armbru@redhat.com>
There is a newer version of this series
block/coroutines.h                         |   4 +-
include/block/block-common.h               |  26 ++++
include/block/block_int-common.h           |  25 ++--
include/block/block_int-io.h               |   4 +-
block/io.c                                 |  51 +++----
block/blkdebug.c                           |   6 +-
block/copy-before-write.c                  |   4 +-
block/file-posix.c                         |  94 +++++++++++--
block/gluster.c                            |   4 +-
block/iscsi.c                              |   6 +-
block/mirror.c                             | 100 +++++++++++---
block/nbd.c                                |   4 +-
block/null.c                               |   6 +-
block/parallels.c                          |   6 +-
block/qcow.c                               |   2 +-
block/qcow2.c                              |   6 +-
block/qed.c                                |   6 +-
block/quorum.c                             |   4 +-
block/raw-format.c                         |   4 +-
block/rbd.c                                |   6 +-
block/snapshot-access.c                    |   4 +-
block/vdi.c                                |   4 +-
block/vmdk.c                               |   2 +-
block/vpc.c                                |   2 +-
block/vvfat.c                              |   6 +-
tests/unit/test-block-iothread.c           |   2 +-
tests/qemu-iotests/194                     |  15 +-
tests/qemu-iotests/194.out                 |   4 +-
tests/qemu-iotests/tests/mirror-sparse     | 100 ++++++++++++++
tests/qemu-iotests/tests/mirror-sparse.out | 153 +++++++++++++++++++++
30 files changed, 547 insertions(+), 113 deletions(-)
create mode 100755 tests/qemu-iotests/tests/mirror-sparse
create mode 100644 tests/qemu-iotests/tests/mirror-sparse.out
[PATCH 0/6] Make blockdev-mirror dest sparse in more cases
Posted by Eric Blake 5 months, 3 weeks ago
When mirroring images, it makes sense for the destination to be sparse
even if it was not connected with "discard":"unmap"; the only time the
destination should be fully allocated is if the user pre-allocated it,
or if the source was not sparse.

Eric Blake (6):
  mirror: Skip pre-zeroing destination if it is already zero
  file-posix: Allow lseek at offset 0 when !want_zero
  mirror: Skip writing zeroes when target is already zero
  block: Expand block status mode from bool to enum
  file-posix: Recognize blockdev-create file as starting all zero
  tests: Add iotest mirror-sparse for recent patches

 block/coroutines.h                         |   4 +-
 include/block/block-common.h               |  26 ++++
 include/block/block_int-common.h           |  25 ++--
 include/block/block_int-io.h               |   4 +-
 block/io.c                                 |  51 +++----
 block/blkdebug.c                           |   6 +-
 block/copy-before-write.c                  |   4 +-
 block/file-posix.c                         |  94 +++++++++++--
 block/gluster.c                            |   4 +-
 block/iscsi.c                              |   6 +-
 block/mirror.c                             | 100 +++++++++++---
 block/nbd.c                                |   4 +-
 block/null.c                               |   6 +-
 block/parallels.c                          |   6 +-
 block/qcow.c                               |   2 +-
 block/qcow2.c                              |   6 +-
 block/qed.c                                |   6 +-
 block/quorum.c                             |   4 +-
 block/raw-format.c                         |   4 +-
 block/rbd.c                                |   6 +-
 block/snapshot-access.c                    |   4 +-
 block/vdi.c                                |   4 +-
 block/vmdk.c                               |   2 +-
 block/vpc.c                                |   2 +-
 block/vvfat.c                              |   6 +-
 tests/unit/test-block-iothread.c           |   2 +-
 tests/qemu-iotests/194                     |  15 +-
 tests/qemu-iotests/194.out                 |   4 +-
 tests/qemu-iotests/tests/mirror-sparse     | 100 ++++++++++++++
 tests/qemu-iotests/tests/mirror-sparse.out | 153 +++++++++++++++++++++
 30 files changed, 547 insertions(+), 113 deletions(-)
 create mode 100755 tests/qemu-iotests/tests/mirror-sparse
 create mode 100644 tests/qemu-iotests/tests/mirror-sparse.out

-- 
2.49.0
Re: [PATCH 0/6] Make blockdev-mirror dest sparse in more cases
Posted by Eric Blake 5 months, 3 weeks ago
On Thu, Apr 10, 2025 at 08:04:50PM -0500, Eric Blake wrote:
> When mirroring images, it makes sense for the destination to be sparse
> even if it was not connected with "discard":"unmap"; the only time the
> destination should be fully allocated is if the user pre-allocated it,
> or if the source was not sparse.
> 
> Eric Blake (6):
>   mirror: Skip pre-zeroing destination if it is already zero
>   file-posix: Allow lseek at offset 0 when !want_zero
>   mirror: Skip writing zeroes when target is already zero
>   block: Expand block status mode from bool to enum
>   file-posix: Recognize blockdev-create file as starting all zero
>   tests: Add iotest mirror-sparse for recent patches

Responding here to point out that in
https://issues.redhat.com/browse/RHEL-82906, Peter Krempa identified
that it was Nir's earlier patch:

> commit d05ae948cc887054495977855b0859d0d4ab2613
> Author: Nir Soffer <nsoffer@redhat.com>
> Date:   Fri Jun 28 23:20:58 2024 +0300
> 
>     Consider discard option when writing zeros
>     
>     When opening an image with discard=off, we punch hole in the image when
>     writing zeroes, making the image sparse. This breaks users that want to
>     ensure that writes cannot fail with ENOSPACE by using fully allocated
>     images[1].
>     
>     bdrv_co_pwrite_zeroes() correctly disables BDRV_REQ_MAY_UNMAP if we
>     opened the child without discard=unmap or discard=on. But we don't go
>     through this function when accessing the top node. Move the check down
>     to bdrv_co_do_pwrite_zeroes() which seems to be used in all code paths.
>     
>     This change implements the documented behavior, punching holes only when
>     opening the image with discard=on or discard=unmap. This may not be the
>     best default but can improve it later.
> ...

that changed qemu's behavior last year to be closer to its documentation.

It raises the question: should we change the default for "discard"
from "ignore" (what we currently have, where write zeroes defaults to
full allocation if you didn't request otherwise - but where this patch
series demonstrates that we can still be careful to avoid writing
zeroes to something that already reads as zeroes as a way to preserve
sparseness) to "unmap" (ie. we would favor sparse files by default,
but where you can still opt-in to full allocation)?

What are the policies on changing defaults?  Do we have to issue a
deprecation notice for any block device opened without specifying a
"discard" policy, and go through a couple of release cycles, so that
two releases down the road we can change the "discard" parameter from
optional to mandatory, and then even later switch it back to optional
but with a new default?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org