From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741216012; cv=none; d=zohomail.com; s=zohoarc; b=UwL08SsYbCe42eZWketGuJgihc9zLNxKPZGS/9Mt9MnLMNedCwNvOndZeXUes4EPlfQBRFMyc1qshiIklNqJHd3NV+B1GT1jdBA8SQLxijgen9dJi7eSsKukSf3k3iaelOn+Ca4JKtj/OSUOfu90daPYaSE4tiisUHl3hdHyXnk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741216012; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=v0aMqzc1F0dHi5tCg3272uegy17iK1O0O6GbkFKQ1uI=; b=CDmLj2os8Vy2P7yPpSsOlp1r2JTJmUTGMLIUrIZC+/fQFX7X5E6rvGF8Wqkk8yFoiSp3innWKEUNDDcikzMMOIS8a8iGVRUgs5kM+FUjF1yWtc2MxAdNoWnZ5EDVxag7DPrjehykkRW88j757lQKD166GXXNPCt/qUOFb7kepX0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741216012547139.2766461755259; Wed, 5 Mar 2025 15:06:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tpxof-0007OQ-M2; Wed, 05 Mar 2025 18:06:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoY-0007N1-WD for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoX-0000Hj-2E for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:10 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-675-_W5aEhsjOKOW8ri72rDrtA-1; Wed, 05 Mar 2025 18:05:50 -0500 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 10F38180049D; Wed, 5 Mar 2025 23:05:49 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.105]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 268021955DCE; Wed, 5 Mar 2025 23:05:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741215967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v0aMqzc1F0dHi5tCg3272uegy17iK1O0O6GbkFKQ1uI=; b=RMu+Tx8+mNjwAbjV8Hj17M+q8DH48urH6QivQkhH13NdMKyBNEBucf6NLTRcjwQX9RMyTg xyJZnLKSbBRouX451DeKJGtbF0P1ORxzlXwutfcNynTzrdqy9oZ6h4cX/bPDqersBPYH7d 7emrjTgDWug/G1tx1I/QbZTi/byJa2s= X-MC-Unique: _W5aEhsjOKOW8ri72rDrtA-1 X-Mimecast-MFC-AGG-ID: _W5aEhsjOKOW8ri72rDrtA_1741215949 From: Eric Blake To: qemu-devel@nongnu.org Cc: Nir Soffer , Stefan Hajnoczi , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org (open list:Block layer core) Subject: [PULL 1/4] iotest: Unbreak 302 with python 3.13 Date: Wed, 5 Mar 2025 17:05:21 -0600 Message-ID: <20250305230542.2225013-7-eblake@redhat.com> In-Reply-To: <20250305230542.2225013-6-eblake@redhat.com> References: <20250305230542.2225013-6-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741216016798019100 Content-Type: text/plain; charset="utf-8" From: Nir Soffer This test depends on TarFile.addfile() to add tar member header without writing the member data, which we write ourself using qemu-nbd. Python 3.13 changed the function in a backward incompatible way[1] to require a file object for tarinfo with non-zero size, breaking the test: -[{"name": "vm.ovf", "offset": 512, "size": 6}, {"name": "disk", "offs= et": 1536, "size": 393216}] +Traceback (most recent call last): + File "/home/stefanha/qemu/tests/qemu-iotests/302", line 118, in + tar.addfile(disk) + ~~~~~~~~~~~^^^^^^ + File "/usr/lib64/python3.13/tarfile.py", line 2262, in addfile + raise ValueError("fileobj not provided for non zero-size regular = file") +ValueError: fileobj not provided for non zero-size regular file The new behavior makes sense for most users, but breaks our unusual usage. Fix the test to add the member header directly using public but undocumented attributes. This is more fragile but the test works again. This also fixes a bug in the previous code - when calling addfile() without a fileobject, tar.offset points to the start of the member data instead of the end. [1] https://github.com/python/cpython/pull/117988 Signed-off-by: Nir Soffer Message-ID: <20250228195708.48035-1-nirsof@gmail.com> Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Signed-off-by: Eric Blake --- tests/qemu-iotests/302 | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302 index a6d79e727b5..e980ec513f2 100755 --- a/tests/qemu-iotests/302 +++ b/tests/qemu-iotests/302 @@ -115,13 +115,22 @@ with tarfile.open(tar_file, "w") as tar: disk =3D tarfile.TarInfo("disk") disk.size =3D actual_size - tar.addfile(disk) - # 6. Shrink the tar to the actual size, aligned to 512 bytes. + # Since python 3.13 we cannot use addfile() to create the member heade= r. + # Add the tarinfo directly using public but undocumented attributes. - tar_size =3D offset + (disk.size + 511) & ~511 - tar.fileobj.seek(tar_size) - tar.fileobj.truncate(tar_size) + buf =3D disk.tobuf(tar.format, tar.encoding, tar.errors) + tar.fileobj.write(buf) + tar.members.append(disk) + + # Update the offset and position to the location of the next member. + + tar.offset =3D offset + (disk.size + 511) & ~511 + tar.fileobj.seek(tar.offset) + + # 6. Shrink the tar to the actual size. + + tar.fileobj.truncate(tar.offset) with tarfile.open(tar_file) as tar: members =3D [{"name": m.name, "size": m.size, "offset": m.offset_data} --=20 2.48.1 From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741216012; cv=none; d=zohomail.com; s=zohoarc; b=AFlIvrBCubETHH/ENNx08JWgzeSbQxzJCn+/hLku8kbYKYpMsnspjUYXjUp14S2QxRVtZkttiUYZ5Sg8yIhrRSZhM9Cz73tRd8JRlj5aTu3yvtkaEHVclgDSnzk31Uv2wwwZFmVxeWL4fJxwWT61tNkO2pWdRZnDaTMOmw0bkDM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741216012; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6CCYjAiR2HYdV55MJcpESwkQpowegStqk4yTWsNqOxo=; b=cX4U0oo1EIbXLy/zQ1Ki8gbxiqWBj6Cxn9BW/EpO1hKY4X/dFzOhZRL7maHjlF4fSbSw8Tyxlm+HuTv1JQU+m0dYHG0w2lbkvNZcFVbaS8N+OM5ahB6w14XWRT7zboRlXjfV6ehbT1XSh8GudjZ9eD+ErAGzlhGmdHXyxBujoQI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741216012610724.4813346766193; Wed, 5 Mar 2025 15:06:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tpxob-0007Mx-2h; Wed, 05 Mar 2025 18:06:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoU-0007Lz-FW for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoS-0000EE-Ul for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:06 -0500 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-443-wYdRhcj1MTuznVzeH2PoVg-1; Wed, 05 Mar 2025 18:05:51 -0500 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E193C180087E; Wed, 5 Mar 2025 23:05:50 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.105]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9B6561955DCE; Wed, 5 Mar 2025 23:05:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741215963; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6CCYjAiR2HYdV55MJcpESwkQpowegStqk4yTWsNqOxo=; b=IxSkLzyn4f8CUkMK6ID5yrFU29q1uX03zya/t7rapvd8i2Iy1oVXgSgeVgC8RtXPX4HfcR gPkbDu8PZcMHK/Z2FScHxt5Ucy6y8IJvMq1NT5E9TN8z9s/zqcBrCug9Ztkk2C3ewisMcv LzLS1/dr9gicMPlpvgmwBuOkP3ZazlE= X-MC-Unique: wYdRhcj1MTuznVzeH2PoVg-1 X-Mimecast-MFC-AGG-ID: wYdRhcj1MTuznVzeH2PoVg_1741215951 From: Eric Blake To: qemu-devel@nongnu.org Cc: Thomas Huth , Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org (open list:Block layer core) Subject: [PULL 2/4] iotests: Stop NBD server in test 162 before starting the next one Date: Wed, 5 Mar 2025 17:05:22 -0600 Message-ID: <20250305230542.2225013-8-eblake@redhat.com> In-Reply-To: <20250305230542.2225013-6-eblake@redhat.com> References: <20250305230542.2225013-6-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741216014789019100 Content-Type: text/plain; charset="utf-8" From: Thomas Huth Test 162 recently started failing for me for no obvious reasons (I did not spot any suspicious commits in this area), but looking in the 162.out.bad log file, there was a suspicious message at the end: qemu-nbd: Cannot lock pid file: Resource temporarily unavailable And indeed, the test starts the NBD server two times, without stopping the first server before running the second one, so the second one can indeed fail to lock the PID file. Thus let's make sure to stop the first server before the test continues with the second one. With this change, the test works fine for me again. Signed-off-by: Thomas Huth Message-ID: <20250225070650.387638-1-thuth@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Eric Blake --- tests/qemu-iotests/162 | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162 index 94dae60d304..956c2c5f339 100755 --- a/tests/qemu-iotests/162 +++ b/tests/qemu-iotests/162 @@ -65,6 +65,7 @@ done $QEMU_IMG info "json:{'driver': 'nbd', 'host': 'localhost', 'port': $port}= " \ | grep '^image' | sed -e "s/$port/PORT/" +_stop_nbd_server # This is a test for NBD's bdrv_refresh_filename() implementation: It expe= cts # either host or path to be set, but it must not assume that they are set = to --=20 2.48.1 From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741216012; cv=none; d=zohomail.com; s=zohoarc; b=Ih9EzMpwe/XCttnq2rK+uspKoEkp9oFTiwFioT8ycVNhAG2H8V1RcQ4c+zNjzfgYZdSV7IGrzTiepsK+mm7PVDV/94FLFRh80GZFztSnasB/a7s+FVPZVekLLauGPCKa4yhLc6ZjEVM4gxxFIsybptn7ouZz8GGc4eSA7BEVh6I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741216012; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5y/cYejfVUQnD5+YFuefcNA8OwV8O2KncbnWN0ret9E=; b=XEGrOG/PTQl+KSmi8lCDQ5aL2jT7JC356i0TAR0Db1bWmhm5+0+ojE2iJUAv2KCzMuRBQcHIM6ALurAlbv718+fWihD5mFzWRp2X9HRr2C6cArGmZqiVpUPCM0JapFSmTHj22rKKuMrMZKIy8P2/PCqTuMPxN9uSuWnQgKqCcU4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741216012607390.89365307341313; Wed, 5 Mar 2025 15:06:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tpxoR-0007Iw-4v; Wed, 05 Mar 2025 18:06:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoO-0007Gz-LP for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoM-0000A7-GD for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:00 -0500 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-8WRBXni6NquSb_HHq6oSIQ-1; Wed, 05 Mar 2025 18:05:54 -0500 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2AC6019560B4; Wed, 5 Mar 2025 23:05:53 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.105]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 69E0F1955DCE; Wed, 5 Mar 2025 23:05:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741215955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5y/cYejfVUQnD5+YFuefcNA8OwV8O2KncbnWN0ret9E=; b=DL8myni7IVEWTNk9C9GokplU6HkzmPu5LmRX5kK54BUDM2GC519nh787jry8UUfNBxBEvh vBokg+u8pr+X6JsKbwjdtxRbQbnmRwetoZPglmuTz8cb+WIBfvMrDeCsaEd5T5AhIGwod4 HiZjcpYJW5NATiFGvTSKJtdwL7Vvu7w= X-MC-Unique: 8WRBXni6NquSb_HHq6oSIQ-1 X-Mimecast-MFC-AGG-ID: 8WRBXni6NquSb_HHq6oSIQ_1741215953 From: Eric Blake To: qemu-devel@nongnu.org Cc: Vladimir Sementsov-Ogievskiy , Kevin Wolf , Hanna Reitz , Markus Armbruster , qemu-block@nongnu.org (open list:Network Block Dev...) Subject: [PULL 3/4] qapi: merge common parts of NbdServerOptions and nbd-server-start data Date: Wed, 5 Mar 2025 17:05:23 -0600 Message-ID: <20250305230542.2225013-9-eblake@redhat.com> In-Reply-To: <20250305230542.2225013-6-eblake@redhat.com> References: <20250305230542.2225013-6-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741216014974019100 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Instead of comment "Keep this type consistent with the nbd-server-start arguments", we can simply merge these things. Note that each field of new base already has "since" tag, equal in both original copies. So "since" information is saved. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-ID: <20250219191914.440451-1-vsementsov@yandex-team.ru> Reviewed-by: Eric Blake Signed-off-by: Eric Blake --- qapi/block-export.json | 94 +++++++++++++++++++----------------------- blockdev-nbd.c | 4 +- 2 files changed, 44 insertions(+), 54 deletions(-) diff --git a/qapi/block-export.json b/qapi/block-export.json index 68dcec7edc5..c783e01a532 100644 --- a/qapi/block-export.json +++ b/qapi/block-export.json @@ -9,53 +9,7 @@ { 'include': 'block-core.json' } ## -# @NbdServerOptions: -# -# Keep this type consistent with the nbd-server-start arguments. The -# only intended difference is using SocketAddress instead of -# SocketAddressLegacy. -# -# @addr: Address on which to listen. -# -# @handshake-max-seconds: Time limit, in seconds, at which a client -# that has not completed the negotiation handshake will be -# disconnected, 0 for no limit (since 10.0; default: 10). -# -# @tls-creds: ID of the TLS credentials object (since 2.6). -# -# @tls-authz: ID of the QAuthZ authorization object used to validate -# the client's x509 distinguished name. This object is is only -# resolved at time of use, so can be deleted and recreated on the -# fly while the NBD server is active. If missing, it will default -# to denying access (since 4.0). -# -# @max-connections: The maximum number of connections to allow at the -# same time, 0 for unlimited. Setting this to 1 also stops the -# server from advertising multiple client support (since 5.2; -# default: 100) -# -# Since: 4.2 -## -{ 'struct': 'NbdServerOptions', - 'data': { 'addr': 'SocketAddress', - '*handshake-max-seconds': 'uint32', - '*tls-creds': 'str', - '*tls-authz': 'str', - '*max-connections': 'uint32' } } - -## -# @nbd-server-start: -# -# Start an NBD server listening on the given host and port. Block -# devices can then be exported using @nbd-server-add. The NBD server -# will present them as named exports; for example, another QEMU -# instance could refer to them as "nbd:HOST:PORT:exportname=3DNAME". -# -# Keep this type consistent with the NbdServerOptions type. The only -# intended difference is using SocketAddressLegacy instead of -# SocketAddress. -# -# @addr: Address on which to listen. +# @NbdServerOptionsBase: # # @handshake-max-seconds: Time limit, in seconds, at which a client # that has not completed the negotiation handshake will be @@ -73,6 +27,46 @@ # same time, 0 for unlimited. Setting this to 1 also stops the # server from advertising multiple client support (since 5.2; # default: 100). +## +{ 'struct': 'NbdServerOptionsBase', + 'data': { '*handshake-max-seconds': 'uint32', + '*tls-creds': 'str', + '*tls-authz': 'str', + '*max-connections': 'uint32' } } + +## +# @NbdServerOptions: +# +# Keep this type consistent with the NbdServerOptionsLegacy type. The +# only intended difference is using SocketAddress instead of +# SocketAddressLegacy. +# +# @addr: Address on which to listen (since 4.2). +## +{ 'struct': 'NbdServerOptions', + 'base': 'NbdServerOptionsBase', + 'data': { 'addr': 'SocketAddress' } } + +## +# @NbdServerOptionsLegacy: +# +# Keep this type consistent with the NbdServerOptions type. The only +# intended difference is using SocketAddressLegacy instead of +# SocketAddress. +# +# @addr: Address on which to listen (since 1.3). +## +{ 'struct': 'NbdServerOptionsLegacy', + 'base': 'NbdServerOptionsBase', + 'data': { 'addr': 'SocketAddressLegacy' } } + +## +# @nbd-server-start: +# +# Start an NBD server listening on the given host and port. Block +# devices can then be exported using @nbd-server-add. The NBD server +# will present them as named exports; for example, another QEMU +# instance could refer to them as "nbd:HOST:PORT:exportname=3DNAME". # # Errors: # - if the server is already running @@ -80,11 +74,7 @@ # Since: 1.3 ## { 'command': 'nbd-server-start', - 'data': { 'addr': 'SocketAddressLegacy', - '*handshake-max-seconds': 'uint32', - '*tls-creds': 'str', - '*tls-authz': 'str', - '*max-connections': 'uint32' }, + 'data': 'NbdServerOptionsLegacy', 'allow-preconfig': true } ## diff --git a/blockdev-nbd.c b/blockdev-nbd.c index 3f6f4ef92b4..1e3e634b87d 100644 --- a/blockdev-nbd.c +++ b/blockdev-nbd.c @@ -219,12 +219,12 @@ void nbd_server_start_options(NbdServerOptions *arg, = Error **errp) arg->tls_authz, arg->max_connections, errp); } -void qmp_nbd_server_start(SocketAddressLegacy *addr, - bool has_handshake_max_secs, +void qmp_nbd_server_start(bool has_handshake_max_secs, uint32_t handshake_max_secs, const char *tls_creds, const char *tls_authz, bool has_max_connections, uint32_t max_connectio= ns, + SocketAddressLegacy *addr, Error **errp) { SocketAddress *addr_flat =3D socket_address_flatten(addr); --=20 2.48.1 From nobody Wed Apr 2 13:35:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1741216012; cv=none; d=zohomail.com; s=zohoarc; b=eaTRlOVYWZ+byE/e+7gZkPOVohfC6j2qLdGCUVZPwsI45AFc6xvDPtCmHTZGHCfAGcCEfMEFoUflrNPS0aPOKClfI9WE/bAyEPtf1TjMn8UV8w9fz7weN6mm5vSHnB0tZJ0JUTl7aDJTIIBfgn9RT3MNb3eSr0cS9r49DK/3CSs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741216012; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=i9l/lI8uB4ISWuy9MK5PFCYsUvJZEkpx1O7spZoc6BU=; b=aBpa8V4bXC2iZ6L518diBcHddPoKKMn2R8ls8kH2NiqQmtq/VQ5AKo3HQze2upItfiddAvO1SVugw9KJSI8xZ7vyKOtuM+8dnLzvyIxMop49cwdzt0M0lBji0oOaKlCHPTeoN3F4CQl0rydx0U2wgz91xL8/T5jo9MmhKKR2BLw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741216012311305.5838641446975; Wed, 5 Mar 2025 15:06:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tpxoS-0007KZ-4q; Wed, 05 Mar 2025 18:06:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoQ-0007Hg-9f for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpxoO-0000Cq-90 for qemu-devel@nongnu.org; Wed, 05 Mar 2025 18:06:02 -0500 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-271-yU595C9SOA2_9z-j9OTpWA-1; Wed, 05 Mar 2025 18:05:56 -0500 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A717B1801A03; Wed, 5 Mar 2025 23:05:54 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.105]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 726E31955DCE; Wed, 5 Mar 2025 23:05:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741215959; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i9l/lI8uB4ISWuy9MK5PFCYsUvJZEkpx1O7spZoc6BU=; b=fbvozqHw7wzdR2qB1/d4hGdNaGXDvqJWv1GVLVh7v7kKBZnxLfKqZv6Kwxew9Uc1+b4fPq lwn0lqmEnhujV6J5McXcn1R0gfawjgJi7mjcc2KlhN8MeWgCY7Fqzr1IGBp0KgI+W47wP5 gCT5/y7zEYcGrFBBhmUhkCV7QfnUC7A= X-MC-Unique: yU595C9SOA2_9z-j9OTpWA-1 X-Mimecast-MFC-AGG-ID: yU595C9SOA2_9z-j9OTpWA_1741215955 From: Eric Blake To: qemu-devel@nongnu.org Cc: Thomas Huth , Vladimir Sementsov-Ogievskiy , qemu-block@nongnu.org (open list:Network Block Dev...) Subject: [PULL 4/4] nbd: Defer trace init until after daemonization Date: Wed, 5 Mar 2025 17:05:24 -0600 Message-ID: <20250305230542.2225013-10-eblake@redhat.com> In-Reply-To: <20250305230542.2225013-6-eblake@redhat.com> References: <20250305230542.2225013-6-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1741216014567019000 Content-Type: text/plain; charset="utf-8" At least the simple trace backend works by spawning a helper thread, and setting up an atexit() handler that coordinates completion with the helper thread. But since atexit registrations survive fork() but helper threads do not, this means that qemu-nbd configured to use the simple trace will deadlock waiting for a thread that no longer exists when it has daemonized. Better is to follow the example of vl.c: don't call any setup functions that might spawn helper threads until we are in the final process that will be doing the work worth tracing. Tested by configuring with --enable-trace-backends=3Dsimple, then running qemu-nbd --fork --trace=3Dnbd_\*,file=3Dqemu-nbd.trace -f raw -r README.r= st followed by `nbdinfo nbd://localhost`, and observing that the trace file is now created without hanging. Reported-by: Thomas Huth Signed-off-by: Eric Blake Message-ID: <20250227220625.870246-2-eblake@redhat.com> Reviewed-by: Thomas Huth --- qemu-nbd.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/qemu-nbd.c b/qemu-nbd.c index 05b61da51ea..ed5895861bb 100644 --- a/qemu-nbd.c +++ b/qemu-nbd.c @@ -852,10 +852,6 @@ int main(int argc, char **argv) export_name =3D ""; } - if (!trace_init_backends()) { - exit(1); - } - trace_init_file(); qemu_set_log(LOG_TRACE, &error_fatal); socket_activation =3D check_socket_activation(); @@ -1045,6 +1041,18 @@ int main(int argc, char **argv) #endif /* WIN32 */ } + /* + * trace_init must be done after daemonization. Why? Because at + * least the simple backend spins up a helper thread as well as an + * atexit() handler that waits on that thread, but the helper + * thread won't survive a fork, leading to deadlock in the child + * if we initialized pre-fork. + */ + if (!trace_init_backends()) { + exit(1); + } + trace_init_file(); + if (opts.device !=3D NULL && sockpath =3D=3D NULL) { sockpath =3D g_malloc(128); snprintf(sockpath, 128, SOCKET_PATH, basename(opts.device)); --=20 2.48.1