tests/qemu-iotests/302 | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
This test depends on TarFile.addfile() to add tar member header without
writing the member data, which we write ourself using qemu-nbd. Python
3.13 changed the function in a backward incompatible way[1] to require a
file object for tarinfo with non-zero size, breaking the test:
-[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}]
+Traceback (most recent call last):
+ File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module>
+ tar.addfile(disk)
+ ~~~~~~~~~~~^^^^^^
+ File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile
+ raise ValueError("fileobj not provided for non zero-size regular file")
+ValueError: fileobj not provided for non zero-size regular file
The new behavior makes sense for most users, but breaks our unusual
usage. Fix the test to add the member header directly using public but
undocumented attributes. This is more fragile but the test works again.
This also fixes a bug in the previous code - when calling addfile()
without a fileobject, tar.offset points to the start of the member data
instead of the end.
[1] https://github.com/python/cpython/pull/117988
Signed-off-by: Nir Soffer <nirsof@gmail.com>
---
tests/qemu-iotests/302 | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
index a6d79e727b..e980ec513f 100755
--- a/tests/qemu-iotests/302
+++ b/tests/qemu-iotests/302
@@ -115,13 +115,22 @@ with tarfile.open(tar_file, "w") as tar:
disk = tarfile.TarInfo("disk")
disk.size = actual_size
- tar.addfile(disk)
- # 6. Shrink the tar to the actual size, aligned to 512 bytes.
+ # Since python 3.13 we cannot use addfile() to create the member header.
+ # Add the tarinfo directly using public but undocumented attributes.
- tar_size = offset + (disk.size + 511) & ~511
- tar.fileobj.seek(tar_size)
- tar.fileobj.truncate(tar_size)
+ buf = disk.tobuf(tar.format, tar.encoding, tar.errors)
+ tar.fileobj.write(buf)
+ tar.members.append(disk)
+
+ # Update the offset and position to the location of the next member.
+
+ tar.offset = offset + (disk.size + 511) & ~511
+ tar.fileobj.seek(tar.offset)
+
+ # 6. Shrink the tar to the actual size.
+
+ tar.fileobj.truncate(tar.offset)
with tarfile.open(tar_file) as tar:
members = [{"name": m.name, "size": m.size, "offset": m.offset_data}
--
2.39.5 (Apple Git-154)On Fri, Feb 28, 2025 at 09:57:08PM +0200, Nir Soffer wrote:
> This test depends on TarFile.addfile() to add tar member header without
> writing the member data, which we write ourself using qemu-nbd. Python
> 3.13 changed the function in a backward incompatible way[1] to require a
> file object for tarinfo with non-zero size, breaking the test:
>
> -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}]
> +Traceback (most recent call last):
> + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module>
> + tar.addfile(disk)
> + ~~~~~~~~~~~^^^^^^
> + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile
> + raise ValueError("fileobj not provided for non zero-size regular file")
> +ValueError: fileobj not provided for non zero-size regular file
>
> The new behavior makes sense for most users, but breaks our unusual
> usage. Fix the test to add the member header directly using public but
> undocumented attributes. This is more fragile but the test works again.
>
> This also fixes a bug in the previous code - when calling addfile()
> without a fileobject, tar.offset points to the start of the member data
> instead of the end.
>
> [1] https://github.com/python/cpython/pull/117988
>
> Signed-off-by: Nir Soffer <nirsof@gmail.com>
> ---
> tests/qemu-iotests/302 | 19 ++++++++++++++-----
> 1 file changed, 14 insertions(+), 5 deletions(-)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
On Mon, Mar 03, 2025 at 04:41:56PM +0800, Stefan Hajnoczi wrote:
> On Fri, Feb 28, 2025 at 09:57:08PM +0200, Nir Soffer wrote:
> > This test depends on TarFile.addfile() to add tar member header without
> > writing the member data, which we write ourself using qemu-nbd. Python
> > 3.13 changed the function in a backward incompatible way[1] to require a
> > file object for tarinfo with non-zero size, breaking the test:
> >
> > -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}]
> > +Traceback (most recent call last):
> > + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module>
> > + tar.addfile(disk)
> > + ~~~~~~~~~~~^^^^^^
> > + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile
> > + raise ValueError("fileobj not provided for non zero-size regular file")
> > +ValueError: fileobj not provided for non zero-size regular file
> >
> > The new behavior makes sense for most users, but breaks our unusual
> > usage. Fix the test to add the member header directly using public but
> > undocumented attributes. This is more fragile but the test works again.
> >
> > This also fixes a bug in the previous code - when calling addfile()
> > without a fileobject, tar.offset points to the start of the member data
> > instead of the end.
> >
> > [1] https://github.com/python/cpython/pull/117988
> >
> > Signed-off-by: Nir Soffer <nirsof@gmail.com>
> > ---
> > tests/qemu-iotests/302 | 19 ++++++++++++++-----
> > 1 file changed, 14 insertions(+), 5 deletions(-)
>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Added to my NBD queue.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
On Fri, Feb 28, 2025 at 09:57:08PM +0200, Nir Soffer wrote:
> This test depends on TarFile.addfile() to add tar member header without
> writing the member data, which we write ourself using qemu-nbd. Python
> 3.13 changed the function in a backward incompatible way[1] to require a
> file object for tarinfo with non-zero size, breaking the test:
>
> -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offset": 1536, "size": 393216}]
> +Traceback (most recent call last):
> + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in <module>
> + tar.addfile(disk)
> + ~~~~~~~~~~~^^^^^^
> + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile
> + raise ValueError("fileobj not provided for non zero-size regular file")
> +ValueError: fileobj not provided for non zero-size regular file
>
> The new behavior makes sense for most users, but breaks our unusual
> usage. Fix the test to add the member header directly using public but
> undocumented attributes. This is more fragile but the test works again.
>
> This also fixes a bug in the previous code - when calling addfile()
> without a fileobject, tar.offset points to the start of the member data
> instead of the end.
>
> [1] https://github.com/python/cpython/pull/117988
>
> Signed-off-by: Nir Soffer <nirsof@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
> tests/qemu-iotests/302 | 19 ++++++++++++++-----
> 1 file changed, 14 insertions(+), 5 deletions(-)
>
> diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
> index a6d79e727b..e980ec513f 100755
> --- a/tests/qemu-iotests/302
> +++ b/tests/qemu-iotests/302
> @@ -115,13 +115,22 @@ with tarfile.open(tar_file, "w") as tar:
>
> disk = tarfile.TarInfo("disk")
> disk.size = actual_size
> - tar.addfile(disk)
>
> - # 6. Shrink the tar to the actual size, aligned to 512 bytes.
> + # Since python 3.13 we cannot use addfile() to create the member header.
> + # Add the tarinfo directly using public but undocumented attributes.
>
> - tar_size = offset + (disk.size + 511) & ~511
> - tar.fileobj.seek(tar_size)
> - tar.fileobj.truncate(tar_size)
> + buf = disk.tobuf(tar.format, tar.encoding, tar.errors)
> + tar.fileobj.write(buf)
> + tar.members.append(disk)
> +
> + # Update the offset and position to the location of the next member.
> +
> + tar.offset = offset + (disk.size + 511) & ~511
> + tar.fileobj.seek(tar.offset)
> +
> + # 6. Shrink the tar to the actual size.
> +
> + tar.fileobj.truncate(tar.offset)
>
> with tarfile.open(tar_file) as tar:
> members = [{"name": m.name, "size": m.size, "offset": m.offset_data}
> --
> 2.39.5 (Apple Git-154)
>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
© 2016 - 2025 Red Hat, Inc.