1. Patchew
  2. QEMU
  3. [PATCH for-5.1 0/2] qemu-img convert -n: Keep qcow2 v2 target sparse
  4. View patch

[PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n

Kevin Wolf posted 2 patches 5 years, 3 months ago
Maintainers: Kevin Wolf <kwolf@redhat.com>, Max Reitz <mreitz@redhat.com>
There is a newer version of this series
[PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n
Posted by Kevin Wolf 5 years, 3 months ago 
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/122     | 34 ++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/122.out | 17 +++++++++++++++++
 2 files changed, 51 insertions(+)

diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
index dfd1cd05d6..1112fc0730 100755
--- a/tests/qemu-iotests/122
+++ b/tests/qemu-iotests/122
@@ -281,6 +281,40 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
 
 $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
 
+echo
+echo '=== -n to an empty image ==='
+echo
+
+_make_test_img 64M
+
+# Convert with -n, which should not result in a fully allocated image, not even
+# with compat=0.10 (because the target doesn't have a backing file)
+TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=1.1 64M
+$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
+$QEMU_IMG map --output=json "$TEST_IMG".orig
+
+TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=0.10 64M
+$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
+$QEMU_IMG map --output=json "$TEST_IMG".orig
+
+echo
+echo '=== -n to an empty image with a backing file ==='
+echo
+
+_make_test_img 64M
+TEST_IMG="$TEST_IMG".base _make_test_img 64M
+
+# Convert with -n, which should still not result in a fully allocated image for
+# compat=1.1 (because it can use zero clusters), but it should be fully
+# allocated with compat=0.10
+TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=1.1 64M
+$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
+$QEMU_IMG map --output=json "$TEST_IMG".orig
+
+TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=0.10 64M
+$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
+$QEMU_IMG map --output=json "$TEST_IMG".orig
+
 echo
 echo '=== -n -B to an image without a backing file ==='
 echo
diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out
index f1f195ed77..b8028efb1d 100644
--- a/tests/qemu-iotests/122.out
+++ b/tests/qemu-iotests/122.out
@@ -229,6 +229,23 @@ wrote 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 Images are identical.
 
+=== -n to an empty image ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
+
+=== -n to an empty image with a backing file ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": false, "data": true, "offset": 327680}]
+
 === -n -B to an image without a backing file ===
 
 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
-- 
2.25.4
Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n
Posted by Nir Soffer 5 years, 3 months ago 
On Mon, Jul 20, 2020 at 4:18 PM Kevin Wolf <kwolf@redhat.com> wrote:
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  tests/qemu-iotests/122     | 34 ++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/122.out | 17 +++++++++++++++++
>  2 files changed, 51 insertions(+)
>
> diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
> index dfd1cd05d6..1112fc0730 100755
> --- a/tests/qemu-iotests/122
> +++ b/tests/qemu-iotests/122
> @@ -281,6 +281,40 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
>
>  $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
>
> +echo
> +echo '=== -n to an empty image ==='
> +echo
> +
> +_make_test_img 64M

Why make a test image here? We create it again below twice

> +
> +# Convert with -n, which should not result in a fully allocated image, not even
> +# with compat=0.10 (because the target doesn't have a backing file)
> +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=1.1 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig

This looks reversed - "$TEST_IMG".orig is the original image, and "$TEST_IMG"
is the target image. So maybe use "$TEST_IMG".target?

> +
> +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=0.10 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig

Since the only difference is the compat, why not use a loop?

for compat in 0.10 1.1; do
...

> +
> +echo
> +echo '=== -n to an empty image with a backing file ==='
> +echo
> +
> +_make_test_img 64M
> +TEST_IMG="$TEST_IMG".base _make_test_img 64M
> +
> +# Convert with -n, which should still not result in a fully allocated image for
> +# compat=1.1 (because it can use zero clusters), but it should be fully
> +# allocated with compat=0.10
> +TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=1.1 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig

Do we have a real use case for this convert? Doesn't this hide all the
data in the
backing file by data from source?

Assuming source is:

src-top: A0--
dst-bas: --B0

And destination is:

dst-top: ----
dst-bas: CCCC

After the convert we will have:

dst-top: A0B0
dst-bas: CCCC

So entire backing of dst is hidden.

Nir

> +TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=0.10 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig
> +
>  echo
>  echo '=== -n -B to an image without a backing file ==='
>  echo
> diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out
> index f1f195ed77..b8028efb1d 100644
> --- a/tests/qemu-iotests/122.out
> +++ b/tests/qemu-iotests/122.out
> @@ -229,6 +229,23 @@ wrote 65536/65536 bytes at offset 0
>  64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  Images are identical.
>
> +=== -n to an empty image ===
> +
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
> +
> +=== -n to an empty image with a backing file ===
> +
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": false, "data": true, "offset": 327680}]
> +
>  === -n -B to an image without a backing file ===
>
>  Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
> --
> 2.25.4
>
Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n
Posted by Kevin Wolf 5 years, 3 months ago 
Am 20.07.2020 um 16:47 hat Nir Soffer geschrieben:
> On Mon, Jul 20, 2020 at 4:18 PM Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> >  tests/qemu-iotests/122     | 34 ++++++++++++++++++++++++++++++++++
> >  tests/qemu-iotests/122.out | 17 +++++++++++++++++
> >  2 files changed, 51 insertions(+)
> >
> > diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
> > index dfd1cd05d6..1112fc0730 100755
> > --- a/tests/qemu-iotests/122
> > +++ b/tests/qemu-iotests/122
> > @@ -281,6 +281,40 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> >
> >  $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
> >
> > +echo
> > +echo '=== -n to an empty image ==='
> > +echo
> > +
> > +_make_test_img 64M
> 
> Why make a test image here? We create it again below twice

This is a different image because the invocations below change the
TEST_IMG variable.

> > +
> > +# Convert with -n, which should not result in a fully allocated image, not even
> > +# with compat=0.10 (because the target doesn't have a backing file)
> > +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=1.1 64M
> > +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> > +$QEMU_IMG map --output=json "$TEST_IMG".orig
> 
> This looks reversed - "$TEST_IMG".orig is the original image, and
> "$TEST_IMG" is the target image. So maybe use "$TEST_IMG".target?

I'll use .orig for the source and without a suffix for the target (which
are filenames that _cleanup_test_img covers automatically).

> > +
> > +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=0.10 64M
> > +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> > +$QEMU_IMG map --output=json "$TEST_IMG".orig
> 
> Since the only difference is the compat, why not use a loop?
> 
> for compat in 0.10 1.1; do
> ...

Makes sense.

> > +
> > +echo
> > +echo '=== -n to an empty image with a backing file ==='
> > +echo
> > +
> > +_make_test_img 64M
> > +TEST_IMG="$TEST_IMG".base _make_test_img 64M
> > +
> > +# Convert with -n, which should still not result in a fully allocated image for
> > +# compat=1.1 (because it can use zero clusters), but it should be fully
> > +# allocated with compat=0.10
> > +TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=1.1 64M
> > +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> > +$QEMU_IMG map --output=json "$TEST_IMG".orig
> 
> Do we have a real use case for this convert? Doesn't this hide all the
> data in the backing file by data from source?

There is probably no real use case for this. But it has a defined
behaviour and it's always good to cover corner cases with tests so that
unintentional changes can be found (which may potentially affect more
relevant cases, too).

Kevin
Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n
Posted by Max Reitz 5 years, 3 months ago 
On 20.07.20 15:18, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  tests/qemu-iotests/122     | 34 ++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/122.out | 17 +++++++++++++++++
>  2 files changed, 51 insertions(+)
> 
> diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
> index dfd1cd05d6..1112fc0730 100755
> --- a/tests/qemu-iotests/122
> +++ b/tests/qemu-iotests/122
> @@ -281,6 +281,40 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
>  
>  $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
>  
> +echo
> +echo '=== -n to an empty image ==='
> +echo
> +
> +_make_test_img 64M
> +
> +# Convert with -n, which should not result in a fully allocated image, not even
> +# with compat=0.10 (because the target doesn't have a backing file)
> +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=1.1 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig
> +
> +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=0.10 64M

It’s a shame that with this, the test will no longer pass with
refcount_bits=1.  (Or an external data file.)

But, well.  Maybe we don’t care and then should just put both options
into _unsupported_imgopts.

Apart from that, the test looks good to me.

Max

> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig
> +
> +echo
> +echo '=== -n to an empty image with a backing file ==='
> +echo
> +
> +_make_test_img 64M
> +TEST_IMG="$TEST_IMG".base _make_test_img 64M
> +
> +# Convert with -n, which should still not result in a fully allocated image for
> +# compat=1.1 (because it can use zero clusters), but it should be fully
> +# allocated with compat=0.10
> +TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=1.1 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig
> +
> +TEST_IMG="$TEST_IMG".orig _make_test_img -b "$TEST_IMG".base -F $IMGFMT -o compat=0.10 64M
> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> +$QEMU_IMG map --output=json "$TEST_IMG".orig
> +
>  echo
>  echo '=== -n -B to an image without a backing file ==='
>  echo
> diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out
> index f1f195ed77..b8028efb1d 100644
> --- a/tests/qemu-iotests/122.out
> +++ b/tests/qemu-iotests/122.out
> @@ -229,6 +229,23 @@ wrote 65536/65536 bytes at offset 0
>  64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  Images are identical.
>  
> +=== -n to an empty image ===
> +
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
> +
> +=== -n to an empty image with a backing file ===
> +
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
> +[{ "start": 0, "length": 67108864, "depth": 0, "zero": false, "data": true, "offset": 327680}]
> +
>  === -n -B to an image without a backing file ===
>  
>  Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
>
Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n
Posted by Kevin Wolf 5 years, 3 months ago 
Am 21.07.2020 um 12:19 hat Max Reitz geschrieben:
> On 20.07.20 15:18, Kevin Wolf wrote:
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> >  tests/qemu-iotests/122     | 34 ++++++++++++++++++++++++++++++++++
> >  tests/qemu-iotests/122.out | 17 +++++++++++++++++
> >  2 files changed, 51 insertions(+)
> > 
> > diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
> > index dfd1cd05d6..1112fc0730 100755
> > --- a/tests/qemu-iotests/122
> > +++ b/tests/qemu-iotests/122
> > @@ -281,6 +281,40 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> >  
> >  $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
> >  
> > +echo
> > +echo '=== -n to an empty image ==='
> > +echo
> > +
> > +_make_test_img 64M
> > +
> > +# Convert with -n, which should not result in a fully allocated image, not even
> > +# with compat=0.10 (because the target doesn't have a backing file)
> > +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=1.1 64M
> > +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
> > +$QEMU_IMG map --output=json "$TEST_IMG".orig
> > +
> > +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=0.10 64M
> 
> It’s a shame that with this, the test will no longer pass with
> refcount_bits=1.  (Or an external data file.)

You mean because of the compat=0.10? We already use that in this test
case, however just with "$QEMU_IMG convert" so that $IMGOPTS doesn't
apply.

I guess I could just override $IMGOPTS for this line to get the same
behaviour here and make sure that none of these options are used.

Kevin
Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n
Posted by Max Reitz 5 years, 3 months ago 
On 21.07.20 13:20, Kevin Wolf wrote:
> Am 21.07.2020 um 12:19 hat Max Reitz geschrieben:
>> On 20.07.20 15:18, Kevin Wolf wrote:
>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>> ---
>>>  tests/qemu-iotests/122     | 34 ++++++++++++++++++++++++++++++++++
>>>  tests/qemu-iotests/122.out | 17 +++++++++++++++++
>>>  2 files changed, 51 insertions(+)
>>>
>>> diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
>>> index dfd1cd05d6..1112fc0730 100755
>>> --- a/tests/qemu-iotests/122
>>> +++ b/tests/qemu-iotests/122
>>> @@ -281,6 +281,40 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
>>>  
>>>  $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
>>>  
>>> +echo
>>> +echo '=== -n to an empty image ==='
>>> +echo
>>> +
>>> +_make_test_img 64M
>>> +
>>> +# Convert with -n, which should not result in a fully allocated image, not even
>>> +# with compat=0.10 (because the target doesn't have a backing file)
>>> +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=1.1 64M
>>> +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
>>> +$QEMU_IMG map --output=json "$TEST_IMG".orig
>>> +
>>> +TEST_IMG="$TEST_IMG".orig _make_test_img -o compat=0.10 64M
>>
>> It’s a shame that with this, the test will no longer pass with
>> refcount_bits=1.  (Or an external data file.)
> 
> You mean because of the compat=0.10? We already use that in this test
> case, however just with "$QEMU_IMG convert" so that $IMGOPTS doesn't
> apply.
> 
> I guess I could just override $IMGOPTS for this line to get the same
> behaviour here and make sure that none of these options are used.

Well... Not my favorite, but probably because I just never thought of that.

I suppose it works, so why not, actually.

Max

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.