1. Patchew
  2. QEMU
  3. View series

[PATCH] qemu-img rebase: don't exceed IO_BUF_SIZE in one operation

Alberto Garcia posted 1 patch 1 week ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20251107091834.383781-1-berto@igalia.com
Maintainers: Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>
qemu-img.c                 |  2 +-
tests/qemu-iotests/024     | 46 ++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/024.out | 27 ++++++++++++++++++++++
3 files changed, 74 insertions(+), 1 deletion(-)
[PATCH] qemu-img rebase: don't exceed IO_BUF_SIZE in one operation
Posted by Alberto Garcia 1 week ago 
During a rebase operation data is copied from the backing chain into
the target image using a loop, and each iteration looks for a
contiguous region of allocated data of at most IO_BUF_SIZE (2 MB).

Once that region is found, and in order to avoid partial writes, its
boundaries are extended so they are aligned to the (sub)clusters of
the target image (see commit 12df580b).

This operation can however result in a region that exceeds the maximum
allowed IO_BUF_SIZE, crashing qemu-img.

This can be easily reproduced when the source image has a smaller
cluster size than the target image:

base <- int <- active

$ qemu-img create -f qcow2 base.qcow2 4M
$ qemu-img create -f qcow2 -F qcow2 -b base.qcow2 -o cluster_size=1M int.qcow2
$ qemu-img create -f qcow2 -F qcow2 -b int.qcow2  -o cluster_size=2M active.qcow2
$ qemu-io -c "write -P 0xff 1M 2M" int.qcow2
$ qemu-img rebase -F qcow2 -b base.qcow2 active.qcow2
qemu-img: qemu-img.c:4102: img_rebase: Assertion `written + pnum <= IO_BUF_SIZE' failed.
Aborted

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3174
Fixes: 12df580b3b7f ("qemu-img: rebase: avoid unnecessary COW operations")
Signed-off-by: Alberto Garcia <berto@igalia.com>
---
 qemu-img.c                 |  2 +-
 tests/qemu-iotests/024     | 46 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/024.out | 27 ++++++++++++++++++++++
 3 files changed, 74 insertions(+), 1 deletion(-)

diff --git a/qemu-img.c b/qemu-img.c
index a7791896c1..454da88c73 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -4081,7 +4081,7 @@ static int img_rebase(const img_cmd_t *ccmd, int argc, char **argv)
             n += offset - QEMU_ALIGN_DOWN(offset, write_align);
             offset = QEMU_ALIGN_DOWN(offset, write_align);
             n += QEMU_ALIGN_UP(offset + n, write_align) - (offset + n);
-            n = MIN(n, size - offset);
+            n = MIN(n, MIN(size - offset, IO_BUF_SIZE));
             assert(!bdrv_is_allocated(unfiltered_bs, offset, n, &n_alloc) &&
                    n_alloc == n);
 
diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024
index b29c76e161..b59d825b42 100755
--- a/tests/qemu-iotests/024
+++ b/tests/qemu-iotests/024
@@ -315,6 +315,52 @@ echo
 
 $QEMU_IMG map "$OVERLAY" | _filter_qemu_img_map
 
+# Check that the region to copy to the overlay during a rebase
+# operation does not exceed the I/O buffer size.
+#
+# backing_new <-- backing_old <-- overlay
+#
+# Backing (new): -- -- -- --    <-- Empty image, size 4MB
+# Backing (old):|--|ff|ff|--|   <-- 4 clusters, 1MB each
+# Overlay:      |-- --|-- --|   <-- 2 clusters, 2MB each
+#
+# The data at [1MB, 3MB) must be copied from the old backing image to
+# the overlay. However the rebase code will extend that region to the
+# overlay's (sub)cluster boundaries to avoid CoW (see commit 12df580b).
+# This test checks that IO_BUF_SIZE (2 MB) is taken into account.
+
+echo
+echo "=== Test that the region to copy does not exceed 2MB (IO_BUF_SIZE) ==="
+echo
+
+echo "Creating backing chain"
+echo
+
+TEST_IMG=$BASE_NEW _make_test_img 4M
+TEST_IMG=$BASE_OLD CLUSTER_SIZE=1M _make_test_img -b "$BASE_NEW" -F $IMGFMT
+TEST_IMG=$OVERLAY  CLUSTER_SIZE=2M _make_test_img -b "$BASE_OLD" -F $IMGFMT
+
+echo
+echo "Writing data to region [1MB, 3MB)"
+echo
+
+$QEMU_IO "$BASE_OLD" -c "write -P 0xff 1M 2M" | _filter_qemu_io
+
+echo
+echo "Rebasing"
+echo
+
+$QEMU_IMG rebase -b "$BASE_NEW" -F $IMGFMT "$OVERLAY"
+
+echo "Verifying the data"
+echo
+
+$QEMU_IO "$OVERLAY" -c "read -P 0x00  0 1M" | _filter_qemu_io
+$QEMU_IO "$OVERLAY" -c "read -P 0xff 1M 2M" | _filter_qemu_io
+$QEMU_IO "$OVERLAY" -c "read -P 0x00 2M 1M" | _filter_qemu_io
+
+$QEMU_IMG map "$OVERLAY" | _filter_qemu_img_map
+
 echo
 
 # success, all done
diff --git a/tests/qemu-iotests/024.out b/tests/qemu-iotests/024.out
index 3d1e31927a..cc18ee0290 100644
--- a/tests/qemu-iotests/024.out
+++ b/tests/qemu-iotests/024.out
@@ -243,4 +243,31 @@ Offset          Length          File
 0               0x20000         TEST_DIR/subdir/t.IMGFMT
 0x40000         0x20000         TEST_DIR/subdir/t.IMGFMT
 
+=== Test that the region to copy does not exceed 2MB (IO_BUF_SIZE) ===
+
+Creating backing chain
+
+Formatting 'TEST_DIR/subdir/t.IMGFMT.base_new', fmt=IMGFMT size=4194304
+Formatting 'TEST_DIR/subdir/t.IMGFMT.base_old', fmt=IMGFMT size=4194304 backing_file=TEST_DIR/subdir/t.IMGFMT.base_new backing_fmt=IMGFMT
+Formatting 'TEST_DIR/subdir/t.IMGFMT', fmt=IMGFMT size=4194304 backing_file=TEST_DIR/subdir/t.IMGFMT.base_old backing_fmt=IMGFMT
+
+Writing data to region [1MB, 3MB)
+
+wrote 2097152/2097152 bytes at offset 1048576
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+Rebasing
+
+Verifying the data
+
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 2097152/2097152 bytes at offset 1048576
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Pattern verification failed at offset 2097152, 1048576 bytes
+read 1048576/1048576 bytes at offset 2097152
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset          Length          File
+0               0x400000        TEST_DIR/subdir/t.IMGFMT
+
 *** done
-- 
2.47.3
Re: [PATCH] qemu-img rebase: don't exceed IO_BUF_SIZE in one operation
Posted by Kevin Wolf 1 week ago 
Am 07.11.2025 um 10:18 hat Alberto Garcia geschrieben:
> During a rebase operation data is copied from the backing chain into
> the target image using a loop, and each iteration looks for a
> contiguous region of allocated data of at most IO_BUF_SIZE (2 MB).
> 
> Once that region is found, and in order to avoid partial writes, its
> boundaries are extended so they are aligned to the (sub)clusters of
> the target image (see commit 12df580b).
> 
> This operation can however result in a region that exceeds the maximum
> allowed IO_BUF_SIZE, crashing qemu-img.
> 
> This can be easily reproduced when the source image has a smaller
> cluster size than the target image:
> 
> base <- int <- active
> 
> $ qemu-img create -f qcow2 base.qcow2 4M
> $ qemu-img create -f qcow2 -F qcow2 -b base.qcow2 -o cluster_size=1M int.qcow2
> $ qemu-img create -f qcow2 -F qcow2 -b int.qcow2  -o cluster_size=2M active.qcow2
> $ qemu-io -c "write -P 0xff 1M 2M" int.qcow2
> $ qemu-img rebase -F qcow2 -b base.qcow2 active.qcow2
> qemu-img: qemu-img.c:4102: img_rebase: Assertion `written + pnum <= IO_BUF_SIZE' failed.
> Aborted
> 
> Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3174
> Fixes: 12df580b3b7f ("qemu-img: rebase: avoid unnecessary COW operations")
> Signed-off-by: Alberto Garcia <berto@igalia.com>
> ---
>  qemu-img.c                 |  2 +-
>  tests/qemu-iotests/024     | 46 ++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/024.out | 27 ++++++++++++++++++++++
>  3 files changed, 74 insertions(+), 1 deletion(-)
> 
> diff --git a/qemu-img.c b/qemu-img.c
> index a7791896c1..454da88c73 100644
> --- a/qemu-img.c
> +++ b/qemu-img.c
> @@ -4081,7 +4081,7 @@ static int img_rebase(const img_cmd_t *ccmd, int argc, char **argv)
>              n += offset - QEMU_ALIGN_DOWN(offset, write_align);
>              offset = QEMU_ALIGN_DOWN(offset, write_align);
>              n += QEMU_ALIGN_UP(offset + n, write_align) - (offset + n);
> -            n = MIN(n, size - offset);
> +            n = MIN(n, MIN(size - offset, IO_BUF_SIZE));
>              assert(!bdrv_is_allocated(unfiltered_bs, offset, n, &n_alloc) &&
>                     n_alloc == n);

The fix itself looks ok.

> +$QEMU_IMG rebase -b "$BASE_NEW" -F $IMGFMT "$OVERLAY"
> +
> +echo "Verifying the data"
> +echo
> +
> +$QEMU_IO "$OVERLAY" -c "read -P 0x00  0 1M" | _filter_qemu_io
> +$QEMU_IO "$OVERLAY" -c "read -P 0xff 1M 2M" | _filter_qemu_io
> +$QEMU_IO "$OVERLAY" -c "read -P 0x00 2M 1M" | _filter_qemu_io

Here you mean 3M instead of 2M...

> +Verifying the data
> +
> +read 1048576/1048576 bytes at offset 0
> +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +read 2097152/2097152 bytes at offset 1048576
> +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +Pattern verification failed at offset 2097152, 1048576 bytes

...which then fixes this failure.

> +read 1048576/1048576 bytes at offset 2097152
> +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +Offset          Length          File
> +0               0x400000        TEST_DIR/subdir/t.IMGFMT
> +
>  *** done

Thanks, I've fixed it up as below and applied the patch to my block
branch.

Kevin

diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024
index b59d825b42..021169b4a1 100755
--- a/tests/qemu-iotests/024
+++ b/tests/qemu-iotests/024
@@ -357,7 +357,7 @@ echo

 $QEMU_IO "$OVERLAY" -c "read -P 0x00  0 1M" | _filter_qemu_io
 $QEMU_IO "$OVERLAY" -c "read -P 0xff 1M 2M" | _filter_qemu_io
-$QEMU_IO "$OVERLAY" -c "read -P 0x00 2M 1M" | _filter_qemu_io
+$QEMU_IO "$OVERLAY" -c "read -P 0x00 3M 1M" | _filter_qemu_io

 $QEMU_IMG map "$OVERLAY" | _filter_qemu_img_map

diff --git a/tests/qemu-iotests/024.out b/tests/qemu-iotests/024.out
index cc18ee0290..1b7522ba71 100644
--- a/tests/qemu-iotests/024.out
+++ b/tests/qemu-iotests/024.out
@@ -264,8 +264,7 @@ read 1048576/1048576 bytes at offset 0
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 2097152/2097152 bytes at offset 1048576
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Pattern verification failed at offset 2097152, 1048576 bytes
-read 1048576/1048576 bytes at offset 2097152
+read 1048576/1048576 bytes at offset 3145728
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 Offset          Length          File
 0               0x400000        TEST_DIR/subdir/t.IMGFMT
Re: [PATCH] qemu-img rebase: don't exceed IO_BUF_SIZE in one operation
Posted by Kevin Wolf 1 week ago 
Am 07.11.2025 um 13:31 hat Kevin Wolf geschrieben:
> Am 07.11.2025 um 10:18 hat Alberto Garcia geschrieben:
> > During a rebase operation data is copied from the backing chain into
> > the target image using a loop, and each iteration looks for a
> > contiguous region of allocated data of at most IO_BUF_SIZE (2 MB).
> > 
> > Once that region is found, and in order to avoid partial writes, its
> > boundaries are extended so they are aligned to the (sub)clusters of
> > the target image (see commit 12df580b).
> > 
> > This operation can however result in a region that exceeds the maximum
> > allowed IO_BUF_SIZE, crashing qemu-img.
> > 
> > This can be easily reproduced when the source image has a smaller
> > cluster size than the target image:
> > 
> > base <- int <- active
> > 
> > $ qemu-img create -f qcow2 base.qcow2 4M
> > $ qemu-img create -f qcow2 -F qcow2 -b base.qcow2 -o cluster_size=1M int.qcow2
> > $ qemu-img create -f qcow2 -F qcow2 -b int.qcow2  -o cluster_size=2M active.qcow2
> > $ qemu-io -c "write -P 0xff 1M 2M" int.qcow2
> > $ qemu-img rebase -F qcow2 -b base.qcow2 active.qcow2
> > qemu-img: qemu-img.c:4102: img_rebase: Assertion `written + pnum <= IO_BUF_SIZE' failed.
> > Aborted
> > 
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3174
> > Fixes: 12df580b3b7f ("qemu-img: rebase: avoid unnecessary COW operations")
> > Signed-off-by: Alberto Garcia <berto@igalia.com>

Oops, I forgot to mention the other point I had:

Cc: qemu-stable <qemu-stable@nongnu.org>

Added it to the Cc: list of this reply and also to the commit message.

Kevin

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.