[Qemu-devel] [PATCH 0/2] qemu-img: Special post-backing convert handle

Max Reitz posted 2 patches 7 years, 6 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20180501165750.19242-1-mreitz@redhat.com
Test checkpatch passed
Test docker-build@min-glib passed
Test docker-mingw@fedora passed
Test s390x passed
qemu-img.c                 | 26 +++++++++++++++++++++++++-
tests/qemu-iotests/122     | 42 ++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/122.out | 18 ++++++++++++++++++
3 files changed, 85 insertions(+), 1 deletion(-)
[Qemu-devel] [PATCH 0/2] qemu-img: Special post-backing convert handle
Posted by Max Reitz 7 years, 6 months ago
This is the issue:

$ qemu-img create -f qcow2 base.qcow2 512M
Formatting 'base.qcow2', fmt=qcow2 size=536870912 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-img create -f qcow2 -b base.qcow2 source.qcow2 1G
Formatting 'source.qcow2', fmt=qcow2 size=1073741824 backing_file=base.qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-img convert -O qcow2 -B base.qcow2 -o compat=0.10 \
    source.qcow2 target.qcow2
$ qemu-img info target.qcow2
image: target.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 512M    <-------------------- this here
cluster_size: 65536
backing file: base.qcow2
Format specific information:
    compat: 0.10
    refcount bits: 16
$ qemu-img map target.qcow2
Offset          Length          Mapped to       File
0x20000000      0x20000000      0x50000         target.qcow2

So qemu-img convert sees that source.qcow2 contains only zeroes past the
end of base.qcow2 -- but that is not the backing file, so it will
explicitly write zeroes to target.qcow2.  But that file is compat=0.10,
so it does not support efficient zero writes and will actually fill that
area with real zeroes.  Hence the mapping, hence the disk size.

However, we don't need to write zeroes to an image when it is
initialized to zeroes -- which the current qemu-img code says doesn't
happen for target images with backing files.  But it does happen when
the target backing file is shorter than the target image, then the area
past the end of the backing file may indeed read as zeroes and we don't
need to write anything there.

So the first patch in this series makes qemu-img convert detect that
case and handle zeroes past the end of the backing file as "fall back to
the backing file" (because that means zeroes, at least if unallocated
areas read as zeroes (which I presume they have to for formats
supporting backing files, but better be safe than sorry)), and the
second adds an iotest.


Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1527898


Max Reitz (2):
  qemu-img: Special post-backing convert handling
  iotests: Test post-backing convert target behavior

 qemu-img.c                 | 26 +++++++++++++++++++++++++-
 tests/qemu-iotests/122     | 42 ++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/122.out | 18 ++++++++++++++++++
 3 files changed, 85 insertions(+), 1 deletion(-)

-- 
2.14.3


Re: [Qemu-devel] [PATCH 0/2] qemu-img: Special post-backing convert handle
Posted by Max Reitz 7 years, 4 months ago
On 2018-05-01 18:57, Max Reitz wrote:
> This is the issue:
> 
> $ qemu-img create -f qcow2 base.qcow2 512M
> Formatting 'base.qcow2', fmt=qcow2 size=536870912 cluster_size=65536 lazy_refcounts=off refcount_bits=16
> $ qemu-img create -f qcow2 -b base.qcow2 source.qcow2 1G
> Formatting 'source.qcow2', fmt=qcow2 size=1073741824 backing_file=base.qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
> $ qemu-img convert -O qcow2 -B base.qcow2 -o compat=0.10 \
>     source.qcow2 target.qcow2
> $ qemu-img info target.qcow2
> image: target.qcow2
> file format: qcow2
> virtual size: 1.0G (1073741824 bytes)
> disk size: 512M    <-------------------- this here
> cluster_size: 65536
> backing file: base.qcow2
> Format specific information:
>     compat: 0.10
>     refcount bits: 16
> $ qemu-img map target.qcow2
> Offset          Length          Mapped to       File
> 0x20000000      0x20000000      0x50000         target.qcow2
> 
> So qemu-img convert sees that source.qcow2 contains only zeroes past the
> end of base.qcow2 -- but that is not the backing file, so it will
> explicitly write zeroes to target.qcow2.  But that file is compat=0.10,
> so it does not support efficient zero writes and will actually fill that
> area with real zeroes.  Hence the mapping, hence the disk size.
> 
> However, we don't need to write zeroes to an image when it is
> initialized to zeroes -- which the current qemu-img code says doesn't
> happen for target images with backing files.  But it does happen when
> the target backing file is shorter than the target image, then the area
> past the end of the backing file may indeed read as zeroes and we don't
> need to write anything there.
> 
> So the first patch in this series makes qemu-img convert detect that
> case and handle zeroes past the end of the backing file as "fall back to
> the backing file" (because that means zeroes, at least if unallocated
> areas read as zeroes (which I presume they have to for formats
> supporting backing files, but better be safe than sorry)), and the
> second adds an iotest.
> 
> 
> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1527898
> 
> 
> Max Reitz (2):
>   qemu-img: Special post-backing convert handling
>   iotests: Test post-backing convert target behavior
> 
>  qemu-img.c                 | 26 +++++++++++++++++++++++++-
>  tests/qemu-iotests/122     | 42 ++++++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/122.out | 18 ++++++++++++++++++
>  3 files changed, 85 insertions(+), 1 deletion(-)

Applied to my block branch.

Max