Add the test case to the iotest #214 that checks possibility of writing
compressed data of more than one cluster size.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
 tests/qemu-iotests/214     | 35 +++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/214.out | 15 +++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index 21ec8a2..0003dc2 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -89,6 +89,41 @@ _check_test_img -r all
 $QEMU_IO -c "read  -P 0x11  0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir
 $QEMU_IO -c "read  -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir
 
+echo
+echo "=== Write compressed data of multiple clusters ==="
+echo
+cluster_size=0x10000
+_make_test_img 2M -o cluster_size=$cluster_size
+
+echo "Uncompressed data:"
+let data_size="8 * $cluster_size"
+$QEMU_IO -c "write -P 0xaa 0 $data_size" "$TEST_IMG" \
+         2>&1 | _filter_qemu_io | _filter_testdir
+$QEMU_IMG info "$TEST_IMG" | sed -n '/disk size:/ s/^ *//p'
+
+_make_test_img 2M -o cluster_size=$cluster_size
+let data_size="3 * $cluster_size + ($cluster_size >> 1)"
+# Set compress=on. That will align the written data
+# by the cluster size and will write them compressed.
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
+$QEMU_IO -c "write -P 0xbb 0 $data_size" --image-opts \
+         driver=$IMGFMT,compress=on,file.filename=$TEST_IMG \
+         2>&1 | _filter_qemu_io | _filter_testdir
+
+let offset="4 * $cluster_size"
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
+$QEMU_IO -c "write -P 0xcc $offset $data_size" "json:{\
+    'driver': '$IMGFMT',
+    'file': {
+        'driver': 'file',
+        'filename': '$TEST_IMG'
+    },
+    'compress': true
+}" | _filter_qemu_io | _filter_testdir
+
+echo "After the multiple cluster data have been written compressed,"
+$QEMU_IMG info "$TEST_IMG" | sed -n '/disk size:/ s/^ *//p'
+
 # success, all done
 echo '*** done'
 rm -f $seq.full
diff --git a/tests/qemu-iotests/214.out b/tests/qemu-iotests/214.out
index 0fcd8dc..09a2e9a 100644
--- a/tests/qemu-iotests/214.out
+++ b/tests/qemu-iotests/214.out
@@ -32,4 +32,19 @@ read 4194304/4194304 bytes at offset 0
 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 4194304/4194304 bytes at offset 4194304
 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+=== Write compressed data of multiple clusters ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
+Uncompressed data:
+wrote 524288/524288 bytes at offset 0
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+disk size: 772 KiB
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
+wrote 229376/229376 bytes at offset 0
+224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 229376/229376 bytes at offset 262144
+224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+After the multiple cluster data have been written compressed,
+disk size: 268 KiB
 *** done
-- 
1.8.3.1

16.10.2019 19:28, Andrey Shinkevich wrote:
> Add the test case to the iotest #214 that checks possibility of writing
> compressed data of more than one cluster size.
> 
> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
> ---
>   tests/qemu-iotests/214     | 35 +++++++++++++++++++++++++++++++++++
>   tests/qemu-iotests/214.out | 15 +++++++++++++++
>   2 files changed, 50 insertions(+)
> 
> diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
> index 21ec8a2..0003dc2 100755
> --- a/tests/qemu-iotests/214
> +++ b/tests/qemu-iotests/214
> @@ -89,6 +89,41 @@ _check_test_img -r all
>   $QEMU_IO -c "read  -P 0x11  0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir
>   $QEMU_IO -c "read  -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir
>   
> +echo
> +echo "=== Write compressed data of multiple clusters ==="
> +echo
> +cluster_size=0x10000
> +_make_test_img 2M -o cluster_size=$cluster_size
> +
> +echo "Uncompressed data:"
> +let data_size="8 * $cluster_size"
> +$QEMU_IO -c "write -P 0xaa 0 $data_size" "$TEST_IMG" \
> +         2>&1 | _filter_qemu_io | _filter_testdir
> +$QEMU_IMG info "$TEST_IMG" | sed -n '/disk size:/ s/^ *//p'

hmm, I'm afraid, "disk size" is unstable thing for iotests. Consider
backporting: other size of metadata in qcow2 image, and we'll have to
fix this test often..

If you want to check, that clusters are compressed, you may use
qemu-img check (with or without --output=json), it shows amount of
compressed clusters.

> +
> +_make_test_img 2M -o cluster_size=$cluster_size
> +let data_size="3 * $cluster_size + ($cluster_size >> 1)"

I'd be a bit more happy with $cluster_size / 2, still, happiness and
bash scripts are incompatible anyway.

> +# Set compress=on. That will align the written data
> +# by the cluster size and will write them compressed.
> +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
> +$QEMU_IO -c "write -P 0xbb 0 $data_size" --image-opts \
> +         driver=$IMGFMT,compress=on,file.filename=$TEST_IMG \
> +         2>&1 | _filter_qemu_io | _filter_testdir
> +
> +let offset="4 * $cluster_size"
> +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
> +$QEMU_IO -c "write -P 0xcc $offset $data_size" "json:{\
> +    'driver': '$IMGFMT',
> +    'file': {
> +        'driver': 'file',
> +        'filename': '$TEST_IMG'
> +    },
> +    'compress': true
> +}" | _filter_qemu_io | _filter_testdir
> +

It would be good to add case with unaligned offset as well. And, maybe,
check that we don't rewrite existing data in partial clusters when writing
unaligned compressed data over it.

> +echo "After the multiple cluster data have been written compressed,"
> +$QEMU_IMG info "$TEST_IMG" | sed -n '/disk size:/ s/^ *//p'
> +
>   # success, all done
>   echo '*** done'
>   rm -f $seq.full
> diff --git a/tests/qemu-iotests/214.out b/tests/qemu-iotests/214.out
> index 0fcd8dc..09a2e9a 100644
> --- a/tests/qemu-iotests/214.out
> +++ b/tests/qemu-iotests/214.out
> @@ -32,4 +32,19 @@ read 4194304/4194304 bytes at offset 0
>   4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>   read 4194304/4194304 bytes at offset 4194304
>   4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +
> +=== Write compressed data of multiple clusters ===
> +
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
> +Uncompressed data:
> +wrote 524288/524288 bytes at offset 0
> +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +disk size: 772 KiB
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
> +wrote 229376/229376 bytes at offset 0
> +224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 229376/229376 bytes at offset 262144
> +224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +After the multiple cluster data have been written compressed,
> +disk size: 268 KiB
>   *** done
> 


-- 
Best regards,
Vladimir

>> +# Set compress=on. That will align the written data
>> +# by the cluster size and will write them compressed.
>> +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
>> +$QEMU_IO -c "write -P 0xbb 0 $data_size" --image-opts \
>> +         driver=$IMGFMT,compress=on,file.filename=$TEST_IMG \
>> +         2>&1 | _filter_qemu_io | _filter_testdir
>> +
>> +let offset="4 * $cluster_size"
>> +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
>> +$QEMU_IO -c "write -P 0xcc $offset $data_size" "json:{\
>> +    'driver': '$IMGFMT',
>> +    'file': {
>> +        'driver': 'file',
>> +        'filename': '$TEST_IMG'
>> +    },
>> +    'compress': true
>> +}" | _filter_qemu_io | _filter_testdir
>> +
> 
> It would be good to add case with unaligned offset as well. And, maybe,"
> check that we don't rewrite existing data in partial clusters when writing
> unaligned compressed data over it.
> 

The I/O error is raised in that case (see 
qcow2_alloc_compressed_cluster_offset()):
"Compression can't overwrite anything. Fail if the cluster was already 
allocated."

#0  qcow2_alloc_compressed_cluster_offset (bs=0x564669143390, 
offset=393216, compressed_size=79, host_offset=0x7f45289d9f00) at 
block/qcow2-cluster.c:767
#1  0x000056466703da7f in qcow2_co_pwritev_compressed_task 
(bs=0x564669143390, offset=393216, bytes=65536, qiov=0x7f4528dddec0, 
qiov_offset=196608) at block/qcow2.c:4198
#2  0x000056466703dc0c in qcow2_co_pwritev_compressed_task_entry 
(task=0x564669152ac0) at block/qcow2.c:4230
#3  0x00005646670a69d0 in aio_task_co (opaque=0x564669152ac0) at 
block/aio_task.c:45
#4  0x0000564667147a87 in coroutine_trampoline (i0=1762994976, i1=22086) 
at util/coroutine-ucontext.c:115

Andrey
-- 
With the best regards,
Andrey Shinkevich