From nobody Sun Apr 28 23:55:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1643048527198815.6561139934431; Mon, 24 Jan 2022 10:22:07 -0800 (PST) Received: from localhost ([::1]:46860 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nC3yg-0001WJ-E5 for importer@patchew.org; Mon, 24 Jan 2022 13:22:06 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38110) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3k0-0001fB-MF for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:06:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60976) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3jz-0006w9-4t for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:06:56 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-436-mJr6ICmgM4OKY9e7Z560pw-1; Mon, 24 Jan 2022 13:06:51 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F11EB1091DA1; Mon, 24 Jan 2022 18:06:49 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.35.175]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3AF38798D3; Mon, 24 Jan 2022 18:06:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643047614; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A/mTcAI+E9s1ILNVU06TWCJz0PZeCT3M7YC3ZqPyCj8=; b=T4IQPRM/f9xMgrcTf1RDjzjv8k0RhHyYqCTxEzwUQnaPibwqVAeeeQM4Z6D7bCEEbTR8Oi Cu63s9b5FL2KuEgRMI3bVFaQ8EZJ/V2AjPH0TT+nC3lSnty4R+RaF6WprrI8VyW9wkvtkX 8Jajp2C/01PevpASsn7OJFQT50rL27s= X-MC-Unique: mJr6ICmgM4OKY9e7Z560pw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 1/4] python/aqmp: Fix negotiation with pre-"oob" QEMU Date: Mon, 24 Jan 2022 13:06:23 -0500 Message-Id: <20220124180626.627718-2-jsnow@redhat.com> In-Reply-To: <20220124180626.627718-1-jsnow@redhat.com> References: <20220124180626.627718-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.158, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Peter Maydell , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , Kevin Wolf , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1643048529373100001 Content-Type: text/plain; charset="utf-8" QEMU versions prior to the "oob" capability *also* can't accept the "enable" keyword argument at all. Fix the handshake process with older QEMU versions. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/aqmp/qmp_client.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py index f1a845cc82..90a8737f03 100644 --- a/python/qemu/aqmp/qmp_client.py +++ b/python/qemu/aqmp/qmp_client.py @@ -292,9 +292,9 @@ async def _negotiate(self) -> None: """ self.logger.debug("Negotiating capabilities ...") =20 - arguments: Dict[str, List[str]] =3D {'enable': []} + arguments: Dict[str, List[str]] =3D {} if self._greeting and 'oob' in self._greeting.QMP.capabilities: - arguments['enable'].append('oob') + arguments.setdefault('enable', []).append('oob') msg =3D self.make_execute_msg('qmp_capabilities', arguments=3Dargu= ments) =20 # It's not safe to use execute() here, because the reader/writers --=20 2.31.1 From nobody Sun Apr 28 23:55:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1643049344955462.8942718785049; Mon, 24 Jan 2022 10:35:44 -0800 (PST) Received: from localhost ([::1]:48558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nC4Bs-0004s0-1m for importer@patchew.org; Mon, 24 Jan 2022 13:35:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3k7-0001qX-Qe for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:07:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42912) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3k6-0006wn-4X for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:07:03 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-130-K_YQIYq0N9qWA9pqH2CZgQ-1; Mon, 24 Jan 2022 13:06:58 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2C24585EE70; Mon, 24 Jan 2022 18:06:57 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.35.175]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9F2C798D3; Mon, 24 Jan 2022 18:06:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643047621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=erCpUhOqiatPqnAj8F0/tsFcIpHI+GWpdKXMz01vZks=; b=WA5wy/3JWQulJuA78ZzIRMwf12+UpRCnZ/1z1iUmv6DukAMYcE0gHkBsQz9Vd0EzAuUZux EXWI9oxsyXBPN6wczb9Q/3Mcsz5LQXtins8iH3ir0UcvN3fZKEE8GLaHdvzhW0Xi30F8Iz Mc+kdQusCYup+jy92KtIfaUtuHAx2HE= X-MC-Unique: K_YQIYq0N9qWA9pqH2CZgQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 2/4] python/machine: raise VMLaunchFailure exception from launch() Date: Mon, 24 Jan 2022 13:06:24 -0500 Message-Id: <20220124180626.627718-3-jsnow@redhat.com> In-Reply-To: <20220124180626.627718-1-jsnow@redhat.com> References: <20220124180626.627718-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.158, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Peter Maydell , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , Kevin Wolf , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1643049347070100001 Content-Type: text/plain; charset="utf-8" This allows us to pack in some extra information about the failure, which guarantees that if the caller did not *intentionally* cause a failure (by capturing this Exception), some pretty good clues will be printed at the bottom of the traceback information. This will help make failures in the event of a non-negative return code more obvious when they go unhandled; the current behavior is to print a warning message only in the event of signal-based terminations (for negative return codes). (Note: In Python, catching BaseException instead of Exception is like installing a signal handler that will run as long as Python itself doesn't crash. KeyboardInterrupt and several other "strong" events in Python are a BaseException. These events should generally not be suppressed, but occasionally we want to perform some cleanup in response to one.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 45 ++++++++++++++++++++--- tests/qemu-iotests/tests/mirror-top-perms | 3 +- 2 files changed, 40 insertions(+), 8 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 67ab06ca2b..a5972fab4d 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -74,6 +74,35 @@ class QEMUMachineAddDeviceError(QEMUMachineError): """ =20 =20 +class VMLaunchFailure(QEMUMachineError): + """ + Exception raised when a VM launch was attempted, but failed. + """ + def __init__(self, exitcode: Optional[int], + command: str, output: Optional[str]): + super().__init__(exitcode, command, output) + self.exitcode =3D exitcode + self.command =3D command + self.output =3D output + + def __str__(self) -> str: + ret =3D '' + if self.__cause__ is not None: + name =3D type(self.__cause__).__name__ + reason =3D str(self.__cause__) + if reason: + ret +=3D f"{name}: {reason}" + else: + ret +=3D f"{name}" + ret +=3D '\n' + + if self.exitcode is not None: + ret +=3D f"\tExit code: {self.exitcode}\n" + ret +=3D f"\tCommand: {self.command}\n" + ret +=3D f"\tOutput: {self.output}\n" + return ret + + class AbnormalShutdown(QEMUMachineError): """ Exception raised when a graceful shutdown was requested, but not perfo= rmed. @@ -397,7 +426,7 @@ def launch(self) -> None: =20 try: self._launch() - except: + except BaseException as exc: # We may have launched the process but it may # have exited before we could connect via QMP. # Assume the VM didn't launch or is exiting. @@ -408,11 +437,15 @@ def launch(self) -> None: else: self._post_shutdown() =20 - LOG.debug('Error launching VM') - if self._qemu_full_args: - LOG.debug('Command: %r', ' '.join(self._qemu_full_args)) - if self._iolog: - LOG.debug('Output: %r', self._iolog) + if isinstance(exc, Exception): + raise VMLaunchFailure( + exitcode=3Dself.exitcode(), + command=3D' '.join(self._qemu_full_args), + output=3Dself._iolog + ) from exc + + # Don't wrap 'BaseException'; doing so would downgrade + # that exception. However, we still want to clean up. raise =20 def _launch(self) -> None: diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index 0a51a613f3..b5849978c4 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -21,7 +21,6 @@ =20 import os =20 -from qemu.aqmp import ConnectError from qemu.machine import machine from qemu.qmp import QMPConnectError =20 @@ -107,7 +106,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.launch() print('ERROR: VM B launched successfully, ' 'this should not have happened') - except (QMPConnectError, ConnectError): + except (QMPConnectError, machine.VMLaunchFailure): assert 'Is another process using the image' in self.vm_b.get_l= og() =20 result =3D self.vm.qmp('block-job-cancel', --=20 2.31.1 From nobody Sun Apr 28 23:55:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1643048806584557.31672749746; Mon, 24 Jan 2022 10:26:46 -0800 (PST) Received: from localhost ([::1]:56314 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nC43B-0007x8-JJ for importer@patchew.org; Mon, 24 Jan 2022 13:26:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38212) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3kP-0002Dp-37 for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:07:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:47476) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3kM-0006xh-IA for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:07:20 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-83-_d_6LLErM_SHd8b5uccqwg-1; Mon, 24 Jan 2022 13:07:14 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BCD5E85EE60; Mon, 24 Jan 2022 18:07:13 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.35.175]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4A4B7A445; Mon, 24 Jan 2022 18:06:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643047638; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VXX7Vk2kcVpctLjnOkCQub6hzrm+bjQjMSo2N13dJT0=; b=h0NjcTkD8LkBjO63okqViyksmwznHzjXdA4zUoMxUI/8U6pgOyneR/B6Wlx4Ub3GLgycHe +WmWuABpPWsdMvwUO3OfV3Pdb01Djp6tB+q/6msBfH73dITdZDUqT+ph1DYmt3KObtuXxy VZHKG979OkWlp5Wd7HKJ2R86E34er3I= X-MC-Unique: _d_6LLErM_SHd8b5uccqwg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 3/4] python: upgrade mypy to 0.780 Date: Mon, 24 Jan 2022 13:06:25 -0500 Message-Id: <20220124180626.627718-4-jsnow@redhat.com> In-Reply-To: <20220124180626.627718-1-jsnow@redhat.com> References: <20220124180626.627718-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.158, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Peter Maydell , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , Kevin Wolf , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1643048808441100001 Content-Type: text/plain; charset="utf-8" We need a slightly newer version of mypy in order to use some features of the asyncio server functions in the next commit. (Note: pipenv is not really suited to upgrading individual packages; I need to replace this tool with something better for the task. For now, the miscellaneous updates not related to the mypy upgrade are simply beyond my control. It's on my list to take care of soon.) Signed-off-by: John Snow --- python/Pipfile.lock | 66 ++++++++++++++++++++++++++------------------- python/setup.cfg | 2 +- 2 files changed, 40 insertions(+), 28 deletions(-) diff --git a/python/Pipfile.lock b/python/Pipfile.lock index d2a7dbd88b..ce46404ce0 100644 --- a/python/Pipfile.lock +++ b/python/Pipfile.lock @@ -1,7 +1,7 @@ { "_meta": { "hash": { - "sha256": "784b327272db32403d5a488507853b5afba850ba26a5948e5b6= a90c1baef2d9c" + "sha256": "f1a25654d884a5b450e38d78b1f2e3ebb9073e421cc4358d4bb= b83ac251a5670" }, "pipfile-spec": 6, "requires": { @@ -34,7 +34,7 @@ "sha256:09bdb456e02564731f8b5957cdd0c98a7f01d2db5e90eb1d79= 4c353c28bfd705", "sha256:6a8a51f64dae307f6e0c9db752b66a7951e282389d8362cc1d= 39a56f3feeb31d" ], - "markers": "python_version ~=3D '3.6'", + "index": "pypi", "version": "=3D=3D2.6.0" }, "avocado-framework": { @@ -50,6 +50,7 @@ "sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa284= 20463a6f799736", "sha256:23e223426b28491b1ced97dc3bbe183027419dfc7982b4fa2f= 05d5f3ff10711c" ], + "index": "pypi", "version": "=3D=3D0.3.2" }, "filelock": { @@ -57,6 +58,7 @@ "sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0= f82159d6a6ff59", "sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10d= a9b879fef15836" ], + "index": "pypi", "version": "=3D=3D3.0.12" }, "flake8": { @@ -88,7 +90,7 @@ "sha256:54161657e8ffc76596c4ede7080ca68cb02962a2e074a2586b= 695a93a925d36e", "sha256:e962bff7440364183203d179d7ae9ad90cb1f2b74dcb84300e= 88ecc42dca3351" ], - "markers": "python_version < '3.7'", + "index": "pypi", "version": "=3D=3D5.1.4" }, "isort": { @@ -124,7 +126,7 @@ "sha256:ed361bb83436f117f9917d282a456f9e5009ea12fd6de8742d= 1a4752c3017e93", "sha256:f5144c75445ae3ca2057faac03fda5a902eff196702b0a24da= f1d6ce0650514b" ], - "markers": "python_version >=3D '2.7' and python_version not i= n '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'", + "index": "pypi", "version": "=3D=3D1.6.0" }, "mccabe": { @@ -136,23 +138,23 @@ }, "mypy": { "hashes": [ - "sha256:15b948e1302682e3682f11f50208b726a246ab4e6c1b39f926= 4a8796bb416aa2", - "sha256:219a3116ecd015f8dca7b5d2c366c973509dfb9a8fc97ef044= a36e3da66144a1", - "sha256:3b1fc683fb204c6b4403a1ef23f0b1fac8e4477091585e0c8c= 54cbdf7d7bb164", - "sha256:3beff56b453b6ef94ecb2996bea101a08f1f8a9771d3cbf498= 8a61e4d9973761", - "sha256:7687f6455ec3ed7649d1ae574136835a4272b65b3ddcf01ab8= 704ac65616c5ce", - "sha256:7ec45a70d40ede1ec7ad7f95b3c94c9cf4c186a32f6bacb179= 5b60abd2f9ef27", - "sha256:86c857510a9b7c3104cf4cde1568f4921762c8f9842e987bc0= 3ed4f160925754", - "sha256:8a627507ef9b307b46a1fea9513d5c98680ba09591253082b4= c48697ba05a4ae", - "sha256:8dfb69fbf9f3aeed18afffb15e319ca7f8da9642336348ddd6= cab2713ddcf8f9", - "sha256:a34b577cdf6313bf24755f7a0e3f3c326d5c1f4fe7422d1d06= 498eb25ad0c600", - "sha256:a8ffcd53cb5dfc131850851cc09f1c44689c2812d0beb954d8= 138d4f5fc17f65", - "sha256:b90928f2d9eb2f33162405f32dde9f6dcead63a0971ca8a1b5= 0eb4ca3e35ceb8", - "sha256:c56ffe22faa2e51054c5f7a3bc70a370939c2ed4de308c690e= 7949230c995913", - "sha256:f91c7ae919bbc3f96cd5e5b2e786b2b108343d1d7972ea130f= 7de27fdd547cf3" + "sha256:00cb1964a7476e871d6108341ac9c1a857d6bd20bf5877f477= 3ac5e9d92cd3cd", + "sha256:127de5a9b817a03a98c5ae8a0c46a20dc44442af6dcfa2ae7f= 96cb519b312efa", + "sha256:1f3976a945ad7f0a0727aafdc5651c2d3278e3c88dee94e2bf= 75cd3386b7b2f4", + "sha256:2f8c098f12b402c19b735aec724cc9105cc1a9eea405d08814= eb4b14a6fb1a41", + "sha256:4ef13b619a289aa025f2273e05e755f8049bb4eaba6d703a42= 5de37d495d178d", + "sha256:5d142f219bf8c7894dfa79ebfb7d352c4c63a325e75f10dfb4= c3db9417dcd135", + "sha256:62eb5dd4ea86bda8ce386f26684f7f26e4bfe6283c9f2b6ca6= d17faf704dcfad", + "sha256:64c36eb0936d0bfb7d8da49f92c18e312ad2e3ed46e5548ae4= ca997b0d33bd59", + "sha256:75eed74d2faf2759f79c5f56f17388defd2fc994222312ec54= ee921e37b31ad4", + "sha256:974bebe3699b9b46278a7f076635d219183da26e1a675c1f82= 43a69221758273", + "sha256:a5e5bb12b7982b179af513dddb06fca12285f0316d74f39640= 78acbfcf4c68f2", + "sha256:d31291df31bafb997952dc0a17ebb2737f802c754aed31dd15= 5a8bfe75112c57", + "sha256:d3b4941de44341227ece1caaf5b08b23e42ad4eeb8b603219a= fb11e9d4cfb437", + "sha256:eadb865126da4e3c4c95bdb47fe1bb087a3e3ea14d39a3b132= 24b8a4d9f9a102" ], "index": "pypi", - "version": "=3D=3D0.770" + "version": "=3D=3D0.780" }, "mypy-extensions": { "hashes": [ @@ -166,7 +168,7 @@ "sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a23037= 4cc1fd776feae5", "sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21= f051443c20947a" ], - "markers": "python_version >=3D '2.7' and python_version not i= n '3.0, 3.1, 3.2, 3.3'", + "index": "pypi", "version": "=3D=3D20.9" }, "pluggy": { @@ -174,7 +176,7 @@ "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604c= f808bfa1c765b0", "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e= 9ca461a4081d2d" ], - "markers": "python_version >=3D '2.7' and python_version not i= n '3.0, 3.1, 3.2, 3.3'", + "index": "pypi", "version": "=3D=3D0.13.1" }, "py": { @@ -182,7 +184,7 @@ "sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a= 809035269e1bd3", "sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d= 02d9d58d18818a" ], - "markers": "python_version >=3D '2.7' and python_version not i= n '3.0, 3.1, 3.2, 3.3'", + "index": "pypi", "version": "=3D=3D1.10.0" }, "pycodestyle": { @@ -205,7 +207,7 @@ "sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15= aef65f0545519f", "sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd= 34d272ca383b8e" ], - "markers": "python_version >=3D '3.5'", + "index": "pypi", "version": "=3D=3D2.9.0" }, "pylint": { @@ -221,13 +223,21 @@ "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ea= d507bc3246ecc1", "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb124= 0a85b024efc88b" ], - "markers": "python_version >=3D '2.6' and python_version not i= n '3.0, 3.1, 3.2, 3.3'", + "index": "pypi", "version": "=3D=3D2.4.7" }, "qemu": { "editable": true, "path": "." }, + "setuptools": { + "hashes": [ + "sha256:22c7348c6d2976a52632c67f7ab0cdf40147db7789f9aed187= 34643fe9cf3373", + "sha256:4ce92f1e1f8f01233ee9952c04f6b81d1e02939d6e1b488428= 154974a4d0783e" + ], + "markers": "python_version >=3D '3.6'", + "version": "=3D=3D59.6.0" + }, "six": { "hashes": [ "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e= 49de9c60c4c926", @@ -294,19 +304,21 @@ "sha256:50b6f157849174217d0656f99dc82fe932884fb250826c1835= 0e159ec6cdf342", "sha256:779383f6086d90c99ae41cf0ff39aac8a7937a9283ce0a414e= 5dd782f4c94a84" ], - "markers": "python_version < '3.8'", + "index": "pypi", "version": "=3D=3D3.10.0.0" }, "urwid": { "hashes": [ "sha256:588bee9c1cb208d0906a9f73c613d2bd32c3ed3702012f51ef= e318a3f2127eae" ], + "index": "pypi", "version": "=3D=3D2.1.2" }, "urwid-readline": { "hashes": [ "sha256:018020cbc864bb5ed87be17dc26b069eae2755cb29f3a9c569= aac3bded1efaf4" ], + "index": "pypi", "version": "=3D=3D0.13" }, "virtualenv": { @@ -314,7 +326,7 @@ "sha256:14fdf849f80dbb29a4eb6caa9875d476ee2a5cf76a5f5415fa= 2f1606010ab467", "sha256:2b0126166ea7c9c3661f5b8e06773d28f83322de7a3ff7d06f= 0aed18c9de6a76" ], - "markers": "python_version >=3D '2.7' and python_version not i= n '3.0, 3.1, 3.2, 3.3'", + "index": "pypi", "version": "=3D=3D20.4.7" }, "wrapt": { @@ -328,7 +340,7 @@ "sha256:3607921face881ba3e026887d8150cca609d517579abe052ac= 81fc5aeffdbd76", "sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea= 5a63edc3e03098" ], - "markers": "python_version < '3.10'", + "index": "pypi", "version": "=3D=3D3.4.1" } } diff --git a/python/setup.cfg b/python/setup.cfg index 3fb18f845d..18aea2bab3 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -41,7 +41,7 @@ devel =3D flake8 >=3D 3.6.0 fusepy >=3D 2.0.4 isort >=3D 5.1.2 - mypy >=3D 0.770 + mypy >=3D 0.780 pylint >=3D 2.8.0 tox >=3D 3.18.0 urwid >=3D 2.1.2 --=20 2.31.1 From nobody Sun Apr 28 23:55:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1643051701165477.32345302859335; Mon, 24 Jan 2022 11:15:01 -0800 (PST) Received: from localhost ([::1]:34284 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nC4nr-0001LX-OD for importer@patchew.org; Mon, 24 Jan 2022 14:14:59 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38848) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3lL-0003m8-2K for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:08:19 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29333) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC3lA-00074x-Db for qemu-devel@nongnu.org; Mon, 24 Jan 2022 13:08:17 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-536-8oNlczPjNeyfy0C29QdVGw-1; Mon, 24 Jan 2022 13:08:00 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 745351018721; Mon, 24 Jan 2022 18:07:59 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.35.175]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1BE8D798D3; Mon, 24 Jan 2022 18:07:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643047681; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kyoRlLPOD1w5OZZS/I5Y/5mwJ+F8Lz40x5dSE6urtqo=; b=JeXlxE/XkOTQPSk7ZMNBuJQBtXHUTRi966yUwvOnPnCBi5QMmnte0s+LKy+74e7LuaRBFw yPQZa+1Pify8L9zPXXQUUA7A3Rm3bhP6rPFyCtkPU8dNqc/EtYE9hi/O6YqYYm8Bv7iJ7s PbEMkcahcrH+6g+WaztrLWnxAEkHWe8= X-MC-Unique: 8oNlczPjNeyfy0C29QdVGw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 4/4] python/aqmp: add socket bind step to legacy.py Date: Mon, 24 Jan 2022 13:06:26 -0500 Message-Id: <20220124180626.627718-5-jsnow@redhat.com> In-Reply-To: <20220124180626.627718-1-jsnow@redhat.com> References: <20220124180626.627718-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.158, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Peter Maydell , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , Kevin Wolf , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1643051703390100001 Content-Type: text/plain; charset="utf-8" The synchronous QMP library would bind to the server address during __init__(). The new library delays this to the accept() call, because binding occurs inside of the call to start_[unix_]server(), which is an async method -- so it cannot happen during __init__ anymore. Python 3.7+ adds the ability to create the server (and thus the bind() call) and begin the active listening in separate steps, but we don't have that functionality in 3.6, our current minimum. Therefore ... Add a temporary workaround that allows the synchronous version of the client to bind the socket in advance, guaranteeing that there will be a UNIX socket in the filesystem ready for the QEMU client to connect to without a race condition. (Yes, it's a bit ugly. Fixing it more nicely will have to wait until our minimum Python version is 3.7+.) Signed-off-by: John Snow --- python/qemu/aqmp/legacy.py | 3 +++ python/qemu/aqmp/protocol.py | 41 +++++++++++++++++++++++++++++++++--- 2 files changed, 41 insertions(+), 3 deletions(-) diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py index 0890f95b16..6baa5f3409 100644 --- a/python/qemu/aqmp/legacy.py +++ b/python/qemu/aqmp/legacy.py @@ -56,6 +56,9 @@ def __init__(self, address: SocketAddrT, self._address =3D address self._timeout: Optional[float] =3D None =20 + if server: + self._aqmp._bind_hack(address) # pylint: disable=3Dprotected-= access + _T =3D TypeVar('_T') =20 def _sync( diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 50e973c2f2..33358f5cd7 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -15,6 +15,7 @@ from enum import Enum from functools import wraps import logging +import socket from ssl import SSLContext from typing import ( Any, @@ -238,6 +239,9 @@ def __init__(self, name: Optional[str] =3D None) -> Non= e: self._runstate =3D Runstate.IDLE self._runstate_changed: Optional[asyncio.Event] =3D None =20 + # Workaround for bind() + self._sock: Optional[socket.socket] =3D None + def __repr__(self) -> str: cls_name =3D type(self).__name__ tokens =3D [] @@ -427,6 +431,34 @@ async def _establish_connection( else: await self._do_connect(address, ssl) =20 + def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None: + """ + Used to create a socket in advance of accept(). + + This is a workaround to ensure that we can guarantee timing of + precisely when a socket exists to avoid a connection attempt + bouncing off of nothing. + + Python 3.7+ adds a feature to separate the server creation and + listening phases instead, and should be used instead of this + hack. + """ + if isinstance(address, tuple): + family =3D socket.AF_INET + else: + family =3D socket.AF_UNIX + + sock =3D socket.socket(family, socket.SOCK_STREAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + + try: + sock.bind(address) + except: + sock.close() + raise + + self._sock =3D sock + @upper_half async def _do_accept(self, address: SocketAddrT, ssl: Optional[SSLContext] =3D None) -> None: @@ -464,24 +496,27 @@ async def _client_connected_cb(reader: asyncio.Stream= Reader, if isinstance(address, tuple): coro =3D asyncio.start_server( _client_connected_cb, - host=3Daddress[0], - port=3Daddress[1], + host=3DNone if self._sock else address[0], + port=3DNone if self._sock else address[1], ssl=3Dssl, backlog=3D1, limit=3Dself._limit, + sock=3Dself._sock, ) else: coro =3D asyncio.start_unix_server( _client_connected_cb, - path=3Daddress, + path=3DNone if self._sock else address, ssl=3Dssl, backlog=3D1, limit=3Dself._limit, + sock=3Dself._sock, ) =20 server =3D await coro # Starts listening await connected.wait() # Waits for the callback to fire (and fini= sh) assert server is None + self._sock =3D None =20 self.logger.debug("Connection accepted.") =20 --=20 2.31.1