From nobody Thu Apr 3 11:33:03 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741216012; cv=none; d=zohomail.com; s=zohoarc; b=UwL08SsYbCe42eZWketGuJgihc9zLNxKPZGS/9Mt9MnLMNedCwNvOndZeXUes4EPlfQBRFMyc1qshiIklNqJHd3NV+B1GT1jdBA8SQLxijgen9dJi7eSsKukSf3k3iaelOn+Ca4JKtj/OSUOfu90daPYaSE4tiisUHl3hdHyXnk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741216012; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=v0aMqzc1F0dHi5tCg3272uegy17iK1O0O6GbkFKQ1uI=; b=CDmLj2os8Vy2P7yPpSsOlp1r2JTJmUTGMLIUrIZC+/fQFX7X5E6rvGF8Wqkk8yFoiSp3innWKEUNDDcikzMMOIS8a8iGVRUgs5kM+FUjF1yWtc2MxAdNoWnZ5EDVxag7DPrjehykkRW88j757lQKD166GXXNPCt/qUOFb7kepX0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741216012547139.2766461755259; Wed, 5 Mar 2025 15:06:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tpxof-0007OQ-M2; Wed, 05 Mar 2025 18:06:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoY-0007N1-WD for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoX-0000Hj-2E for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:10 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-675-_W5aEhsjOKOW8ri72rDrtA-1; Wed, 05 Mar 2025 18:05:50 -0500 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 10F38180049D; Wed, 5 Mar 2025 23:05:49 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.105]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 268021955DCE; Wed, 5 Mar 2025 23:05:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741215967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v0aMqzc1F0dHi5tCg3272uegy17iK1O0O6GbkFKQ1uI=; b=RMu+Tx8+mNjwAbjV8Hj17M+q8DH48urH6QivQkhH13NdMKyBNEBucf6NLTRcjwQX9RMyTg xyJZnLKSbBRouX451DeKJGtbF0P1ORxzlXwutfcNynTzrdqy9oZ6h4cX/bPDqersBPYH7d 7emrjTgDWug/G1tx1I/QbZTi/byJa2s= X-MC-Unique: _W5aEhsjOKOW8ri72rDrtA-1 X-Mimecast-MFC-AGG-ID: _W5aEhsjOKOW8ri72rDrtA_1741215949 From: Eric Blake To: qemu-devel@nongnu.org Cc: Nir Soffer , Stefan Hajnoczi , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org (open list:Block layer core) Subject: [PULL 1/4] iotest: Unbreak 302 with python 3.13 Date: Wed, 5 Mar 2025 17:05:21 -0600 Message-ID: <20250305230542.2225013-7-eblake@redhat.com> In-Reply-To: <20250305230542.2225013-6-eblake@redhat.com> References: <20250305230542.2225013-6-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741216016798019100 Content-Type: text/plain; charset="utf-8" From: Nir Soffer This test depends on TarFile.addfile() to add tar member header without writing the member data, which we write ourself using qemu-nbd. Python 3.13 changed the function in a backward incompatible way[1] to require a file object for tarinfo with non-zero size, breaking the test: -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offs= et": 1536, "size": 393216}] +Traceback (most recent call last): + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in + tar.addfile(disk) + ~~~~~~~~~~~^^^^^^ + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile + raise ValueError("fileobj not provided for non zero-size regular = file") +ValueError: fileobj not provided for non zero-size regular file The new behavior makes sense for most users, but breaks our unusual usage. Fix the test to add the member header directly using public but undocumented attributes. This is more fragile but the test works again. This also fixes a bug in the previous code - when calling addfile() without a fileobject, tar.offset points to the start of the member data instead of the end. [1] https://github.com/python/cpython/pull/117988 Signed-off-by: Nir Soffer Message-ID: <20250228195708.48035-1-nirsof@gmail.com> Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Signed-off-by: Eric Blake --- tests/qemu-iotests/302 | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302 index a6d79e727b5..e980ec513f2 100755 --- a/tests/qemu-iotests/302 +++ b/tests/qemu-iotests/302 @@ -115,13 +115,22 @@ with tarfile.open(tar_file, "w") as tar: disk =3D tarfile.TarInfo("disk") disk.size =3D actual_size - tar.addfile(disk) - # 6. Shrink the tar to the actual size, aligned to 512 bytes. + # Since python 3.13 we cannot use addfile() to create the member heade= r. + # Add the tarinfo directly using public but undocumented attributes. - tar_size =3D offset + (disk.size + 511) & ~511 - tar.fileobj.seek(tar_size) - tar.fileobj.truncate(tar_size) + buf =3D disk.tobuf(tar.format, tar.encoding, tar.errors) + tar.fileobj.write(buf) + tar.members.append(disk) + + # Update the offset and position to the location of the next member. + + tar.offset =3D offset + (disk.size + 511) & ~511 + tar.fileobj.seek(tar.offset) + + # 6. Shrink the tar to the actual size. + + tar.fileobj.truncate(tar.offset) with tarfile.open(tar_file) as tar: members =3D [{"name": m.name, "size": m.size, "offset": m.offset_data} --=20 2.48.1