From nobody Sat May 18 23:55:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691433288164.80024254360626; Mon, 7 Mar 2022 14:17:13 -0800 (PST) Received: from localhost ([::1]:46046 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLfF-0001hO-56 for importer@patchew.org; Mon, 07 Mar 2022 17:17:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47890) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdW-0006I8-73 for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:40672) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-00015M-J0 for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:25 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-28-8KMvTVCeNCqtAjwncmvbzg-1; Mon, 07 Mar 2022 17:15:11 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9EE3A1006AA6; Mon, 7 Mar 2022 22:15:10 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 99F454530F; Mon, 7 Mar 2022 22:15:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691315; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3n3UifIGxSXWkJqViNL9go59/YPxvs0XKpFvj4xHck4=; b=JgianO65vqZNS033FjJmTRDC1bHagkxzP7kLFUqkew9dJbmvhDYRVZwsBpR3Z4v4NTfoug K+NR+DYccgtzkbHLvZHh/qJwBz99Xkd5fjG99ofQRgog6GYS1JoI+9XuOkA1z9TxDpX92h qyChtcDJ8AV9xSAJjRUBmkNqfrh0pPk= X-MC-Unique: 8KMvTVCeNCqtAjwncmvbzg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 01/11] python/aqmp: add _session_guard() Date: Mon, 7 Mar 2022 17:14:57 -0500 Message-Id: <20220307221507.1218892-2-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691435075100001 In _new_session, there's a fairly complex except clause that's used to give semantic errors to callers of accept() and connect(). We need to create a new two-step replacement for accept(), so factoring out this piece of logic will be useful. Bolster the comments and docstring here to try and demystify what's going on in this fairly delicate piece of Python magic. (If we were using Python 3.7+, this would be an @asynccontextmanager. We don't have that very nice piece of magic, however, so this must take an Awaitable to manage the Exception contexts properly. We pay the price for platform compatibility.) Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-2-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 89 +++++++++++++++++++++++++----------- 1 file changed, 62 insertions(+), 27 deletions(-) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 33358f5cd7..009883f64d 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -317,6 +317,62 @@ async def disconnect(self) -> None: # Section: Session machinery # -------------------------- =20 + async def _session_guard(self, coro: Awaitable[None], emsg: str) -> No= ne: + """ + Async guard function used to roll back to `IDLE` on any error. + + On any Exception, the state machine will be reset back to + `IDLE`. Most Exceptions will be wrapped with `ConnectError`, but + `BaseException` events will be left alone (This includes + asyncio.CancelledError, even prior to Python 3.8). + + :param error_message: + Human-readable string describing what connection phase failed. + + :raise BaseException: + When `BaseException` occurs in the guarded block. + :raise ConnectError: + When any other error is encountered in the guarded block. + """ + # Note: After Python 3.6 support is removed, this should be an + # @asynccontextmanager instead of accepting a callback. + try: + await coro + except BaseException as err: + self.logger.error("%s: %s", emsg, exception_summary(err)) + self.logger.debug("%s:\n%s\n", emsg, pretty_traceback()) + try: + # Reset the runstate back to IDLE. + await self.disconnect() + except: + # We don't expect any Exceptions from the disconnect funct= ion + # here, because we failed to connect in the first place. + # The disconnect() function is intended to perform + # only cannot-fail cleanup here, but you never know. + emsg =3D ( + "Unexpected bottom half exception. " + "This is a bug in the QMP library. " + "Please report it to and " + "CC: John Snow ." + ) + self.logger.critical("%s:\n%s\n", emsg, pretty_traceback()) + raise + + # CancelledError is an Exception with special semantic meaning; + # We do NOT want to wrap it up under ConnectError. + # NB: CancelledError is not a BaseException before Python 3.8 + if isinstance(err, asyncio.CancelledError): + raise + + # Any other kind of error can be treated as some kind of conne= ction + # failure broadly. Inspect the 'exc' field to explore the root + # cause in greater detail. + if isinstance(err, Exception): + raise ConnectError(emsg, err) from err + + # Raise BaseExceptions un-wrapped, they're more important. + raise + @property def _runstate_event(self) -> asyncio.Event: # asyncio.Event() objects should not be created prior to entrance = into @@ -371,34 +427,13 @@ async def _new_session(self, """ assert self.runstate =3D=3D Runstate.IDLE =20 - try: - phase =3D "connection" - await self._establish_connection(address, ssl, accept) + await self._session_guard( + self._establish_connection(address, ssl, accept), + 'Failed to establish connection') =20 - phase =3D "session" - await self._establish_session() - - except BaseException as err: - emsg =3D f"Failed to establish {phase}" - self.logger.error("%s: %s", emsg, exception_summary(err)) - self.logger.debug("%s:\n%s\n", emsg, pretty_traceback()) - try: - # Reset from CONNECTING back to IDLE. - await self.disconnect() - except: - emsg =3D "Unexpected bottom half exception" - self.logger.critical("%s:\n%s\n", emsg, pretty_traceback()) - raise - - # NB: CancelledError is not a BaseException before Python 3.8 - if isinstance(err, asyncio.CancelledError): - raise - - if isinstance(err, Exception): - raise ConnectError(emsg, err) from err - - # Raise BaseExceptions un-wrapped, they're more important. - raise + await self._session_guard( + self._establish_session(), + 'Failed to establish session') =20 assert self.runstate =3D=3D Runstate.RUNNING =20 --=20 2.34.1 From nobody Sat May 18 23:55:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691578697713.8830781201016; Mon, 7 Mar 2022 14:19:38 -0800 (PST) Received: from localhost ([::1]:54494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLhZ-0007GV-JB for importer@patchew.org; Mon, 07 Mar 2022 17:19:37 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47948) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdX-0006Ll-7x for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52013) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-00015V-Jr for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:26 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-266-2u8_MaBYPWartU-Xz1afYg-1; Mon, 07 Mar 2022 17:15:12 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C8854800D55; Mon, 7 Mar 2022 22:15:11 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id C31135E253; Mon, 7 Mar 2022 22:15:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691316; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PEAlOhA/D683GPriceZXeZytFE6nN7yOXzcaWji/GYA=; b=Rsxa14H8IYKS0OQUn9VD+9qpsPwZJOnRONBGdUE4UjwnSDvoGD+9uAAsPhlK1r3ZQbUoZl VodmnmVshchC+JDahbuHdtRR/XVC0rvsyZuSOWZcw0F5xenDr6Q0L5roeyJ1XVwz0ggRFF wxDRqEyY9QjmuBDztNHPrI/Cs2RxSME= X-MC-Unique: 2u8_MaBYPWartU-Xz1afYg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 02/11] python/aqmp: rename 'accept()' to 'start_server_and_accept()' Date: Mon, 7 Mar 2022 17:14:58 -0500 Message-Id: <20220307221507.1218892-3-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691580802100001 Previously, I had a method named "accept()" that under-the-hood calls bind(2), listen(2) *and* accept(2). I meant this as a simplification and counterpart to the one-shot "connect()" method. This is confusing to readers who expect accept() to mean *just* accept(2). Since I need to split apart the "accept()" method into multiple methods anyway (one of which strongly resembling accept(2)), it feels pertinent to rename this method *now*. Rename this all-in-one method "start_server_and_accept()" instead. Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-3-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/legacy.py | 2 +- python/qemu/aqmp/protocol.py | 6 ++++-- python/tests/protocol.py | 24 ++++++++++++------------ 3 files changed, 17 insertions(+), 15 deletions(-) diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py index 6baa5f3409..dca1e76ed4 100644 --- a/python/qemu/aqmp/legacy.py +++ b/python/qemu/aqmp/legacy.py @@ -91,7 +91,7 @@ def accept(self, timeout: Optional[float] =3D 15.0) -> QM= PMessage: self._aqmp.negotiate =3D True =20 self._sync( - self._aqmp.accept(self._address), + self._aqmp.start_server_and_accept(self._address), timeout ) =20 diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 009883f64d..73719257e0 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -265,8 +265,10 @@ async def runstate_changed(self) -> Runstate: =20 @upper_half @require(Runstate.IDLE) - async def accept(self, address: SocketAddrT, - ssl: Optional[SSLContext] =3D None) -> None: + async def start_server_and_accept( + self, address: SocketAddrT, + ssl: Optional[SSLContext] =3D None + ) -> None: """ Accept a connection and begin processing message queues. =20 diff --git a/python/tests/protocol.py b/python/tests/protocol.py index 5cd7938be3..354d6559b9 100644 --- a/python/tests/protocol.py +++ b/python/tests/protocol.py @@ -413,14 +413,14 @@ async def _bad_connection(self, family: str): assert family in ('INET', 'UNIX') =20 if family =3D=3D 'INET': - await self.proto.accept(('example.com', 1)) + await self.proto.start_server_and_accept(('example.com', 1)) elif family =3D=3D 'UNIX': - await self.proto.accept('/dev/null') + await self.proto.start_server_and_accept('/dev/null') =20 async def _hanging_connection(self): with TemporaryDirectory(suffix=3D'.aqmp') as tmpdir: sock =3D os.path.join(tmpdir, type(self.proto).__name__ + ".so= ck") - await self.proto.accept(sock) + await self.proto.start_server_and_accept(sock) =20 =20 class FakeSession(TestBase): @@ -449,13 +449,13 @@ async def testFakeConnect(self): @TestBase.async_test async def testFakeAccept(self): """Test the full state lifecycle (via accept) with a no-op session= .""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') self.assertEqual(self.proto.runstate, Runstate.RUNNING) =20 @TestBase.async_test async def testFakeRecv(self): """Test receiving a fake/null message.""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') =20 logname =3D self.proto.logger.name with self.assertLogs(logname, level=3D'DEBUG') as context: @@ -471,7 +471,7 @@ async def testFakeRecv(self): @TestBase.async_test async def testFakeSend(self): """Test sending a fake/null message.""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') =20 logname =3D self.proto.logger.name with self.assertLogs(logname, level=3D'DEBUG') as context: @@ -493,7 +493,7 @@ async def _prod_session_api( ): with self.assertRaises(StateError) as context: if accept: - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path= ') else: await self.proto.connect('/not/a/real/path') =20 @@ -504,7 +504,7 @@ async def _prod_session_api( @TestBase.async_test async def testAcceptRequireRunning(self): """Test that accept() cannot be called when Runstate=3DRUNNING""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') =20 await self._prod_session_api( Runstate.RUNNING, @@ -515,7 +515,7 @@ async def testAcceptRequireRunning(self): @TestBase.async_test async def testConnectRequireRunning(self): """Test that connect() cannot be called when Runstate=3DRUNNING""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') =20 await self._prod_session_api( Runstate.RUNNING, @@ -526,7 +526,7 @@ async def testConnectRequireRunning(self): @TestBase.async_test async def testAcceptRequireDisconnecting(self): """Test that accept() cannot be called when Runstate=3DDISCONNECTI= NG""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') =20 # Cheat: force a disconnect. await self.proto.simulate_disconnect() @@ -541,7 +541,7 @@ async def testAcceptRequireDisconnecting(self): @TestBase.async_test async def testConnectRequireDisconnecting(self): """Test that connect() cannot be called when Runstate=3DDISCONNECT= ING""" - await self.proto.accept('/not/a/real/path') + await self.proto.start_server_and_accept('/not/a/real/path') =20 # Cheat: force a disconnect. await self.proto.simulate_disconnect() @@ -576,7 +576,7 @@ async def _asyncTearDown(self): async def testSmoke(self): with TemporaryDirectory(suffix=3D'.aqmp') as tmpdir: sock =3D os.path.join(tmpdir, type(self.proto).__name__ + ".so= ck") - server_task =3D create_task(self.server.accept(sock)) + server_task =3D create_task(self.server.start_server_and_accep= t(sock)) =20 # give the server a chance to start listening [...] await asyncio.sleep(0) --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691582228841.803533307774; Mon, 7 Mar 2022 14:19:42 -0800 (PST) Received: from localhost ([::1]:54692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLhd-0007OY-5S for importer@patchew.org; Mon, 07 Mar 2022 17:19:41 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdW-0006IC-87 for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25485) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-00015O-Im for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:25 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-184-xXa52afCNNim2Fc1KjbvMA-1; Mon, 07 Mar 2022 17:15:14 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F3E73801DDB; Mon, 7 Mar 2022 22:15:12 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id EEE5A5E253; Mon, 7 Mar 2022 22:15:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691315; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3ENHoTsYi5K2XUbX9eFa0q5Ulpgll9RlmODXIa3EJBs=; b=ihWvvrdmok3ne76qUW2AZH/MgjP94ySP6f6NeUvQUaUc79pdf2HxP2/CjhummBBr3mb/DQ u3DYHmoj7J74bKufD9lcck9gjkH6MxUYnMG18deGc2M6LtSjtBXQfo+BIBbXucA6hzQFZW JmZOrokynX8erRSeWFYWzGjcjmcivnk= X-MC-Unique: xXa52afCNNim2Fc1KjbvMA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 03/11] python/aqmp: remove _new_session and _establish_connection Date: Mon, 7 Mar 2022 17:14:59 -0500 Message-Id: <20220307221507.1218892-4-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691582845100003 These two methods attempted to entirely envelop the logic of establishing a connection to a peer start to finish. However, we need to break apart the incoming connection step into more granular steps. We will no longer be able to reasonably constrain the logic inside of these helper functions. So, remove them - with _session_guard(), they no longer serve a real purpose. Although the public API doesn't change, the internal API does. Now that there are no intermediary methods between e.g. connect() and _do_connect(), there's no hook where the runstate is set. As a result, the test suite changes a little to cope with the new semantics of _do_accept() and _do_connect(). Lastly, take some pieces of the now-deleted docstrings and move them up to the public interface level. They were a little more detailed, and it won't hurt to keep them. Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-4-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 117 ++++++++++++++--------------------- python/tests/protocol.py | 10 ++- 2 files changed, 53 insertions(+), 74 deletions(-) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 73719257e0..b7e5e635d8 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -275,13 +275,25 @@ async def start_server_and_accept( If this call fails, `runstate` is guaranteed to be set back to `ID= LE`. =20 :param address: - Address to listen to; UNIX socket path or TCP address/port. + Address to listen on; UNIX socket path or TCP address/port. :param ssl: SSL context to use, if any. =20 :raise StateError: When the `Runstate` is not `IDLE`. - :raise ConnectError: If a connection could not be accepted. + :raise ConnectError: + When a connection or session cannot be established. + + This exception will wrap a more concrete one. In most cases, + the wrapped exception will be `OSError` or `EOFError`. If a + protocol-level failure occurs while establishing a new + session, the wrapped error may also be an `QMPError`. """ - await self._new_session(address, ssl, accept=3DTrue) + await self._session_guard( + self._do_accept(address, ssl), + 'Failed to establish connection') + await self._session_guard( + self._establish_session(), + 'Failed to establish session') + assert self.runstate =3D=3D Runstate.RUNNING =20 @upper_half @require(Runstate.IDLE) @@ -297,9 +309,21 @@ async def connect(self, address: SocketAddrT, :param ssl: SSL context to use, if any. =20 :raise StateError: When the `Runstate` is not `IDLE`. - :raise ConnectError: If a connection cannot be made to the server. + :raise ConnectError: + When a connection or session cannot be established. + + This exception will wrap a more concrete one. In most cases, + the wrapped exception will be `OSError` or `EOFError`. If a + protocol-level failure occurs while establishing a new + session, the wrapped error may also be an `QMPError`. """ - await self._new_session(address, ssl) + await self._session_guard( + self._do_connect(address, ssl), + 'Failed to establish connection') + await self._session_guard( + self._establish_session(), + 'Failed to establish session') + assert self.runstate =3D=3D Runstate.RUNNING =20 @upper_half async def disconnect(self) -> None: @@ -401,73 +425,6 @@ def _set_state(self, state: Runstate) -> None: self._runstate_event.set() self._runstate_event.clear() =20 - @upper_half - async def _new_session(self, - address: SocketAddrT, - ssl: Optional[SSLContext] =3D None, - accept: bool =3D False) -> None: - """ - Establish a new connection and initialize the session. - - Connect or accept a new connection, then begin the protocol - session machinery. If this call fails, `runstate` is guaranteed - to be set back to `IDLE`. - - :param address: - Address to connect to/listen on; - UNIX socket path or TCP address/port. - :param ssl: SSL context to use, if any. - :param accept: Accept a connection instead of connecting when `Tru= e`. - - :raise ConnectError: - When a connection or session cannot be established. - - This exception will wrap a more concrete one. In most cases, - the wrapped exception will be `OSError` or `EOFError`. If a - protocol-level failure occurs while establishing a new - session, the wrapped error may also be an `QMPError`. - """ - assert self.runstate =3D=3D Runstate.IDLE - - await self._session_guard( - self._establish_connection(address, ssl, accept), - 'Failed to establish connection') - - await self._session_guard( - self._establish_session(), - 'Failed to establish session') - - assert self.runstate =3D=3D Runstate.RUNNING - - @upper_half - async def _establish_connection( - self, - address: SocketAddrT, - ssl: Optional[SSLContext] =3D None, - accept: bool =3D False - ) -> None: - """ - Establish a new connection. - - :param address: - Address to connect to/listen on; - UNIX socket path or TCP address/port. - :param ssl: SSL context to use, if any. - :param accept: Accept a connection instead of connecting when `Tru= e`. - """ - assert self.runstate =3D=3D Runstate.IDLE - self._set_state(Runstate.CONNECTING) - - # Allow runstate watchers to witness 'CONNECTING' state; some - # failures in the streaming layer are synchronous and will not - # otherwise yield. - await asyncio.sleep(0) - - if accept: - await self._do_accept(address, ssl) - else: - await self._do_connect(address, ssl) - def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None: """ Used to create a socket in advance of accept(). @@ -508,6 +465,9 @@ async def _do_accept(self, address: SocketAddrT, =20 :raise OSError: For stream-related errors. """ + assert self.runstate =3D=3D Runstate.IDLE + self._set_state(Runstate.CONNECTING) + self.logger.debug("Awaiting connection on %s ...", address) connected =3D asyncio.Event() server: Optional[asyncio.AbstractServer] =3D None @@ -550,6 +510,11 @@ async def _client_connected_cb(reader: asyncio.StreamR= eader, sock=3Dself._sock, ) =20 + # Allow runstate watchers to witness 'CONNECTING' state; some + # failures in the streaming layer are synchronous and will not + # otherwise yield. + await asyncio.sleep(0) + server =3D await coro # Starts listening await connected.wait() # Waits for the callback to fire (and fini= sh) assert server is None @@ -569,6 +534,14 @@ async def _do_connect(self, address: SocketAddrT, =20 :raise OSError: For stream-related errors. """ + assert self.runstate =3D=3D Runstate.IDLE + self._set_state(Runstate.CONNECTING) + + # Allow runstate watchers to witness 'CONNECTING' state; some + # failures in the streaming layer are synchronous and will not + # otherwise yield. + await asyncio.sleep(0) + self.logger.debug("Connecting to %s ...", address) =20 if isinstance(address, tuple): diff --git a/python/tests/protocol.py b/python/tests/protocol.py index 354d6559b9..8dd26c4ed1 100644 --- a/python/tests/protocol.py +++ b/python/tests/protocol.py @@ -42,11 +42,17 @@ async def _establish_session(self): await super()._establish_session() =20 async def _do_accept(self, address, ssl=3DNone): - if not self.fake_session: + if self.fake_session: + self._set_state(Runstate.CONNECTING) + await asyncio.sleep(0) + else: await super()._do_accept(address, ssl) =20 async def _do_connect(self, address, ssl=3DNone): - if not self.fake_session: + if self.fake_session: + self._set_state(Runstate.CONNECTING) + await asyncio.sleep(0) + else: await super()._do_connect(address, ssl) =20 async def _do_recv(self) -> None: --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691600472651.5322627589887; Mon, 7 Mar 2022 14:20:00 -0800 (PST) Received: from localhost ([::1]:55480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLhu-0007uF-K2 for importer@patchew.org; Mon, 07 Mar 2022 17:20:00 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdW-0006JM-GE for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43794) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-00015l-Jc for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:26 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-500-761H1q4ZOmOK3f8pUFL1SQ-1; Mon, 07 Mar 2022 17:15:15 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2564F51DF; Mon, 7 Mar 2022 22:15:14 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 221355E253; Mon, 7 Mar 2022 22:15:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EX84oLX6c+vG8gpKnoiP7xQjkuvSV2Hr2eFWSJzoge0=; b=Ffemr5pxEl81hJXpifQseGE2Oalkk0gr8qTj85YGaaFsYl6dGJdJTQ6VdWA0abyz1LbWai CB7Z7172N4IeM4NhfbWHLsjsB1ucC+vg11dKhmABt+sx2oQhybMa8u4MqC0GM47oj8F41K l7qwCWdH5DzxqSzfdaB5tXYNGikR1+E= X-MC-Unique: 761H1q4ZOmOK3f8pUFL1SQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 04/11] python/aqmp: split _client_connected_cb() out as _incoming() Date: Mon, 7 Mar 2022 17:15:00 -0500 Message-Id: <20220307221507.1218892-5-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691608184100001 As part of disentangling the monolithic nature of _do_accept(), split out the incoming callback to prepare for factoring out the "wait for a peer" step. Namely, this means using an event signal we can wait on from outside of this method. Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-5-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 83 +++++++++++++++++++++++++----------- 1 file changed, 58 insertions(+), 25 deletions(-) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index b7e5e635d8..56f05b9030 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -242,6 +242,10 @@ def __init__(self, name: Optional[str] =3D None) -> No= ne: # Workaround for bind() self._sock: Optional[socket.socket] =3D None =20 + # Server state for start_server() and _incoming() + self._server: Optional[asyncio.AbstractServer] =3D None + self._accepted: Optional[asyncio.Event] =3D None + def __repr__(self) -> str: cls_name =3D type(self).__name__ tokens =3D [] @@ -425,6 +429,54 @@ def _set_state(self, state: Runstate) -> None: self._runstate_event.set() self._runstate_event.clear() =20 + @bottom_half # However, it does not run from the R/W tasks. + async def _stop_server(self) -> None: + """ + Stop listening for / accepting new incoming connections. + """ + if self._server is None: + return + + try: + self.logger.debug("Stopping server.") + self._server.close() + await self._server.wait_closed() + self.logger.debug("Server stopped.") + finally: + self._server =3D None + + @bottom_half # However, it does not run from the R/W tasks. + async def _incoming(self, + reader: asyncio.StreamReader, + writer: asyncio.StreamWriter) -> None: + """ + Accept an incoming connection and signal the upper_half. + + This method does the minimum necessary to accept a single + incoming connection. It signals back to the upper_half ASAP so + that any errors during session initialization can occur + naturally in the caller's stack. + + :param reader: Incoming `asyncio.StreamReader` + :param writer: Incoming `asyncio.StreamWriter` + """ + peer =3D writer.get_extra_info('peername', 'Unknown peer') + self.logger.debug("Incoming connection from %s", peer) + + if self._reader or self._writer: + # Sadly, we can have more than one pending connection + # because of https://bugs.python.org/issue46715 + # Close any extra connections we don't actually want. + self.logger.warning("Extraneous connection inadvertently accep= ted") + writer.close() + return + + # A connection has been accepted; stop listening for new ones. + assert self._accepted is not None + await self._stop_server() + self._reader, self._writer =3D (reader, writer) + self._accepted.set() + def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None: """ Used to create a socket in advance of accept(). @@ -469,30 +521,11 @@ async def _do_accept(self, address: SocketAddrT, self._set_state(Runstate.CONNECTING) =20 self.logger.debug("Awaiting connection on %s ...", address) - connected =3D asyncio.Event() - server: Optional[asyncio.AbstractServer] =3D None - - async def _client_connected_cb(reader: asyncio.StreamReader, - writer: asyncio.StreamWriter) -> No= ne: - """Used to accept a single incoming connection, see below.""" - nonlocal server - nonlocal connected - - # A connection has been accepted; stop listening for new ones. - assert server is not None - server.close() - await server.wait_closed() - server =3D None - - # Register this client as being connected - self._reader, self._writer =3D (reader, writer) - - # Signal back: We've accepted a client! - connected.set() + self._accepted =3D asyncio.Event() =20 if isinstance(address, tuple): coro =3D asyncio.start_server( - _client_connected_cb, + self._incoming, host=3DNone if self._sock else address[0], port=3DNone if self._sock else address[1], ssl=3Dssl, @@ -502,7 +535,7 @@ async def _client_connected_cb(reader: asyncio.StreamRe= ader, ) else: coro =3D asyncio.start_unix_server( - _client_connected_cb, + self._incoming, path=3DNone if self._sock else address, ssl=3Dssl, backlog=3D1, @@ -515,9 +548,9 @@ async def _client_connected_cb(reader: asyncio.StreamRe= ader, # otherwise yield. await asyncio.sleep(0) =20 - server =3D await coro # Starts listening - await connected.wait() # Waits for the callback to fire (and fini= sh) - assert server is None + self._server =3D await coro # Starts listening + await self._accepted.wait() # Waits for the callback to finish + assert self._server is None self._sock =3D None =20 self.logger.debug("Connection accepted.") --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164669141689440.98318344386587; Mon, 7 Mar 2022 14:16:56 -0800 (PST) Received: from localhost ([::1]:44406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLex-0000ch-Jn for importer@patchew.org; Mon, 07 Mar 2022 17:16:55 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdT-0006FA-7M for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30414) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-00015h-HN for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:22 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-607-mAXcOdphOs6P3K6AYMgTxw-1; Mon, 07 Mar 2022 17:15:16 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4AE631091DA0; Mon, 7 Mar 2022 22:15:15 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4944E5E255; Mon, 7 Mar 2022 22:15:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691317; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YH+julnTCcrOrmUBkLnIFOBpciKKivzmN5lvvLKd+i8=; b=fm8KI8qDzuHzYpRj/9OAYoldhxXu3DVpjQtA44jv8hn7Bg1mSJU9v8vfL1yEubs8zcLaYr Qrid5H4g0vIE9FsrAsildjbCVKIzj05dqBbTcS4OoEON7EUnyz+cm/ZCNpSWbOxChfwrjU CFylFF5bxL/NbQFiL1bntQ1pnwjqdos= X-MC-Unique: mAXcOdphOs6P3K6AYMgTxw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 05/11] python/aqmp: squelch pylint warning for too many lines Date: Mon, 7 Mar 2022 17:15:01 -0500 Message-Id: <20220307221507.1218892-6-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691419726100001 I would really like to keep this under 1000 lines, I promise. Doesn't look like it's gonna happen. Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-6-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 56f05b9030..631bcdaa55 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -10,6 +10,9 @@ class. """ =20 +# It's all the docstrings ... ! It's long for a good reason ^_^; +# pylint: disable=3Dtoo-many-lines + import asyncio from asyncio import StreamReader, StreamWriter from enum import Enum --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691420822471.25818611192153; Mon, 7 Mar 2022 14:17:00 -0800 (PST) Received: from localhost ([::1]:44530 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLf1-0000hk-JD for importer@patchew.org; Mon, 07 Mar 2022 17:16:59 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47820) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdU-0006FR-Ar for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:54435) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-00016A-HU for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:24 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-193-Qicm9lxON9SE0IZuA6asGw-1; Mon, 07 Mar 2022 17:15:17 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9FE0D180FD74; Mon, 7 Mar 2022 22:15:16 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FB9D5E253; Mon, 7 Mar 2022 22:15:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CEaY/I575l6aSE1FzKUgMgmUVQzEt80iip/FhWQmA6E=; b=JDqbd0Q21HrKNoSJFRUMbwh7k1YeSMarHKG6xunZdnUxUN6VqCWtJtzAD0qioU4FhPHb+A xsyC+0yyvmy2sXH3Zs6uSMAyCpOCVa6BMZYpJY6gABdc3krqxaI/NipBWVWdpS5o53qOjA /m9sm0iF4zL8ip38mBxIfYfPM1L1Gx8= X-MC-Unique: Qicm9lxON9SE0IZuA6asGw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 06/11] python/aqmp: refactor _do_accept() into two distinct steps Date: Mon, 7 Mar 2022 17:15:02 -0500 Message-Id: <20220307221507.1218892-7-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691422867100002 Refactor _do_accept() into _do_start_server() and _do_accept(). As of this commit, the former calls the latter, but in subsequent commits they'll be split apart. (So please forgive the misnomer for _do_start_server(); it will live up to its name shortly, and the docstring will be updated then too. I'm just cutting down on some churn.) Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-7-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 29 ++++++++++++++++++++++++----- python/tests/protocol.py | 4 ++-- 2 files changed, 26 insertions(+), 7 deletions(-) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 631bcdaa55..e2bdad542d 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -295,7 +295,7 @@ async def start_server_and_accept( session, the wrapped error may also be an `QMPError`. """ await self._session_guard( - self._do_accept(address, ssl), + self._do_start_server(address, ssl), 'Failed to establish connection') await self._session_guard( self._establish_session(), @@ -509,8 +509,8 @@ def _bind_hack(self, address: Union[str, Tuple[str, int= ]]) -> None: self._sock =3D sock =20 @upper_half - async def _do_accept(self, address: SocketAddrT, - ssl: Optional[SSLContext] =3D None) -> None: + async def _do_start_server(self, address: SocketAddrT, + ssl: Optional[SSLContext] =3D None) -> None: """ Acting as the transport server, accept a single connection. =20 @@ -551,9 +551,28 @@ async def _do_accept(self, address: SocketAddrT, # otherwise yield. await asyncio.sleep(0) =20 - self._server =3D await coro # Starts listening - await self._accepted.wait() # Waits for the callback to finish + # This will start the server (bind(2), listen(2)). It will also + # call accept(2) if we yield, but we don't block on that here. + self._server =3D await coro + + # Just for this one commit, wait for a peer. + # This gets split out in the next patch. + await self._do_accept() + + @upper_half + async def _do_accept(self) -> None: + """ + Wait for and accept an incoming connection. + + Requires that we have not yet accepted an incoming connection + from the upper_half, but it's OK if the server is no longer + running because the bottom_half has already accepted the + connection. + """ + assert self._accepted is not None + await self._accepted.wait() assert self._server is None + self._accepted =3D None self._sock =3D None =20 self.logger.debug("Connection accepted.") diff --git a/python/tests/protocol.py b/python/tests/protocol.py index 8dd26c4ed1..5e442e1efb 100644 --- a/python/tests/protocol.py +++ b/python/tests/protocol.py @@ -41,12 +41,12 @@ async def _establish_session(self): self.trigger_input =3D asyncio.Event() await super()._establish_session() =20 - async def _do_accept(self, address, ssl=3DNone): + async def _do_start_server(self, address, ssl=3DNone): if self.fake_session: self._set_state(Runstate.CONNECTING) await asyncio.sleep(0) else: - await super()._do_accept(address, ssl) + await super()._do_start_server(address, ssl) =20 async def _do_connect(self, address, ssl=3DNone): if self.fake_session: --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691420838730.3643999020724; Mon, 7 Mar 2022 14:17:00 -0800 (PST) Received: from localhost ([::1]:44626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLf1-0000lv-Rn for importer@patchew.org; Mon, 07 Mar 2022 17:16:59 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47822) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdU-0006FS-Pq for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:58363) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdR-000164-Hp for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:24 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-292-FpxexmPQNkmOuVU71-tauw-1; Mon, 07 Mar 2022 17:15:18 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A5BE1801DDB; Mon, 7 Mar 2022 22:15:17 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id A01085E26D; Mon, 7 Mar 2022 22:15:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TrRpSa6wEGYbNC2oBAkag8DkUDwHJlq7gm5Lu56Ykb0=; b=XwZasK8dCNM/3vV3ZZwUvGt9a1/4w+7BwQfRz5WoGxWxz6R8ZifYyhqeQUa12kjexlVsP9 4Pbevru3GAjXLu6J9MJcMQbNTyHTaoV4MgRgShr2y7llAOJ59Uq0Vf6L/xcQcbI0tMn8YL ASv1ZXkYhz8I4u9qcbBuGwLXSLZw4ic= X-MC-Unique: FpxexmPQNkmOuVU71-tauw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 07/11] python/aqmp: stop the server during disconnect() Date: Mon, 7 Mar 2022 17:15:03 -0500 Message-Id: <20220307221507.1218892-8-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691422856100001 Before we allow the full separation of starting the server and accepting new connections, make sure that the disconnect cleans up the server and its new state, too. Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-8-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index e2bdad542d..cdbc9cba0d 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -432,7 +432,7 @@ def _set_state(self, state: Runstate) -> None: self._runstate_event.set() self._runstate_event.clear() =20 - @bottom_half # However, it does not run from the R/W tasks. + @bottom_half async def _stop_server(self) -> None: """ Stop listening for / accepting new incoming connections. @@ -709,6 +709,7 @@ def _paranoid_task_erase(task: Optional['asyncio.Future= [_U]'] =20 self._reader =3D None self._writer =3D None + self._accepted =3D None =20 # NB: _runstate_changed cannot be cleared because we still need it= to # send the final runstate changed event ...! @@ -732,6 +733,9 @@ async def _bh_disconnect(self) -> None: def _done(task: Optional['asyncio.Future[Any]']) -> bool: return task is not None and task.done() =20 + # If the server is running, stop it. + await self._stop_server() + # Are we already in an error pathway? If either of the tasks are # already done, or if we have no tasks but a reader/writer; we # must be. --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691741043499.4468085314363; Mon, 7 Mar 2022 14:22:21 -0800 (PST) Received: from localhost ([::1]:35538 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLkB-00052j-Gw for importer@patchew.org; Mon, 07 Mar 2022 17:22:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48034) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdZ-0006Qx-AP for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:48318) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdT-00016g-J6 for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:28 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-225-TZ8tLnwUP-GxoLZ8nF_CBA-1; Mon, 07 Mar 2022 17:15:20 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D528D51F8; Mon, 7 Mar 2022 22:15:18 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB9695E253; Mon, 7 Mar 2022 22:15:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691323; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E8KbTXbhlSsycVB2+i4RNmY4qy9EQwnMi4MC4red4Ys=; b=L044+yXZuZoU6I1If7WqtdRsK4xyDL0IpD+Jflaaiqt+aTMKcaX5vmefKju2ghZVI/JUku 3BADdotJMLeHyqizrw9jBtMi4GtM3jDuczZnjcjuz1S8YXMuG7ZrTOqy1D80CU9M42HDyV 7RyXTUIWms2xhXfLU1i/ife84KTuRVc= X-MC-Unique: TZ8tLnwUP-GxoLZ8nF_CBA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 08/11] python/aqmp: add start_server() and accept() methods Date: Mon, 7 Mar 2022 17:15:04 -0500 Message-Id: <20220307221507.1218892-9-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691744355100001 Add start_server() and accept() methods that can be used instead of start_server_and_accept() to allow more fine-grained control over the incoming connection process. (Eagle-eyed reviewers will surely notice that it's a bit weird that "CONNECTING" is a state that's shared between both the start_server() and connect() states. That's absolutely true, and it's very true that checking on the presence of _accepted as an indicator of state is a hack. That's also very certainly true. But ... this keeps client code an awful lot simpler, as it doesn't have to care exactly *how* the connection is being made, just that it *is*. Is it worth disrupting that simplicity in order to provide a better state guard on `accept()`? Hm.) Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-9-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 67 +++++++++++++++++++++++++++++++++--- python/tests/protocol.py | 7 ++++ 2 files changed, 69 insertions(+), 5 deletions(-) diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index cdbc9cba0d..2ecba14555 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -280,6 +280,8 @@ async def start_server_and_accept( Accept a connection and begin processing message queues. =20 If this call fails, `runstate` is guaranteed to be set back to `ID= LE`. + This method is precisely equivalent to calling `start_server()` + followed by `accept()`. =20 :param address: Address to listen on; UNIX socket path or TCP address/port. @@ -294,9 +296,62 @@ async def start_server_and_accept( protocol-level failure occurs while establishing a new session, the wrapped error may also be an `QMPError`. """ + await self.start_server(address, ssl) + await self.accept() + assert self.runstate =3D=3D Runstate.RUNNING + + @upper_half + @require(Runstate.IDLE) + async def start_server(self, address: SocketAddrT, + ssl: Optional[SSLContext] =3D None) -> None: + """ + Start listening for an incoming connection, but do not wait for a = peer. + + This method starts listening for an incoming connection, but + does not block waiting for a peer. This call will return + immediately after binding and listening on a socket. A later + call to `accept()` must be made in order to finalize the + incoming connection. + + :param address: + Address to listen on; UNIX socket path or TCP address/port. + :param ssl: SSL context to use, if any. + + :raise StateError: When the `Runstate` is not `IDLE`. + :raise ConnectError: + When the server could not start listening on this address. + + This exception will wrap a more concrete one. In most cases, + the wrapped exception will be `OSError`. + """ await self._session_guard( self._do_start_server(address, ssl), 'Failed to establish connection') + assert self.runstate =3D=3D Runstate.CONNECTING + + @upper_half + @require(Runstate.CONNECTING) + async def accept(self) -> None: + """ + Accept an incoming connection and begin processing message queues. + + If this call fails, `runstate` is guaranteed to be set back to `ID= LE`. + + :raise StateError: When the `Runstate` is not `CONNECTING`. + :raise QMPError: When `start_server()` was not called yet. + :raise ConnectError: + When a connection or session cannot be established. + + This exception will wrap a more concrete one. In most cases, + the wrapped exception will be `OSError` or `EOFError`. If a + protocol-level failure occurs while establishing a new + session, the wrapped error may also be an `QMPError`. + """ + if self._accepted is None: + raise QMPError("Cannot call accept() before start_server().") + await self._session_guard( + self._do_accept(), + 'Failed to establish connection') await self._session_guard( self._establish_session(), 'Failed to establish session') @@ -512,7 +567,12 @@ def _bind_hack(self, address: Union[str, Tuple[str, in= t]]) -> None: async def _do_start_server(self, address: SocketAddrT, ssl: Optional[SSLContext] =3D None) -> None: """ - Acting as the transport server, accept a single connection. + Start listening for an incoming connection, but do not wait for a = peer. + + This method starts listening for an incoming connection, but does = not + block waiting for a peer. This call will return immediately after + binding and listening to a socket. A later call to accept() must be + made in order to finalize the incoming connection. =20 :param address: Address to listen on; UNIX socket path or TCP address/port. @@ -554,10 +614,7 @@ async def _do_start_server(self, address: SocketAddrT, # This will start the server (bind(2), listen(2)). It will also # call accept(2) if we yield, but we don't block on that here. self._server =3D await coro - - # Just for this one commit, wait for a peer. - # This gets split out in the next patch. - await self._do_accept() + self.logger.debug("Server listening on %s", address) =20 @upper_half async def _do_accept(self) -> None: diff --git a/python/tests/protocol.py b/python/tests/protocol.py index 5e442e1efb..d6849ad306 100644 --- a/python/tests/protocol.py +++ b/python/tests/protocol.py @@ -43,11 +43,18 @@ async def _establish_session(self): =20 async def _do_start_server(self, address, ssl=3DNone): if self.fake_session: + self._accepted =3D asyncio.Event() self._set_state(Runstate.CONNECTING) await asyncio.sleep(0) else: await super()._do_start_server(address, ssl) =20 + async def _do_accept(self): + if self.fake_session: + self._accepted =3D None + else: + await super()._do_accept() + async def _do_connect(self, address, ssl=3DNone): if self.fake_session: self._set_state(Runstate.CONNECTING) --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691708205219.3484046154; Mon, 7 Mar 2022 14:21:48 -0800 (PST) Received: from localhost ([::1]:33430 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLjg-0003di-1p for importer@patchew.org; Mon, 07 Mar 2022 17:21:48 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47968) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdX-0006Ny-Qn for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:39297) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdT-00016R-0Y for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:27 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-345-Nm3ajl2xNGW6KGsWf41j1w-1; Mon, 07 Mar 2022 17:15:21 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 07067800D55; Mon, 7 Mar 2022 22:15:20 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 039515E253; Mon, 7 Mar 2022 22:15:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691322; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uX2xYOfks0dxXpJ5arraJ4+qoeIBhRVJhy65dmuNaRM=; b=NULUWy7xjKC4djHgzx35S35PSXBalVWpJg7Z2/rITzocJ0tmCXCTmSftbvaM+bxc2aWWS7 yxkZgy4vxcQhWkRJ8oqf3abgLyk25jynhqN02TrWjfthB4orcdqToPWRBKc0l5txC071Qn 7I23aBlRGi9NLM4CzDwrEDD/liz9aKA= X-MC-Unique: Nm3ajl2xNGW6KGsWf41j1w-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 09/11] python/aqmp: fix race condition in legacy.py Date: Mon, 7 Mar 2022 17:15:05 -0500 Message-Id: <20220307221507.1218892-10-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691709642100001 legacy.py provides a synchronous model. iotests frequently uses this paradigm: - create QMP client object - start QEMU process - await connection from QEMU process In the switch from sync to async QMP, the QMP client object stopped calling bind() and listen() during the QMP object creation step, which creates a race condition if the QEMU process dials in too quickly. With refactoring out of the way, restore the former behavior of calling bind() and listen() during __init__() to fix this race condition. Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-10-jsnow@redhat.com [Expanded commit message. --js] Signed-off-by: John Snow --- python/qemu/aqmp/legacy.py | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py index dca1e76ed4..cb50e60564 100644 --- a/python/qemu/aqmp/legacy.py +++ b/python/qemu/aqmp/legacy.py @@ -57,7 +57,7 @@ def __init__(self, address: SocketAddrT, self._timeout: Optional[float] =3D None =20 if server: - self._aqmp._bind_hack(address) # pylint: disable=3Dprotected-= access + self._sync(self._aqmp.start_server(address)) =20 _T =3D TypeVar('_T') =20 @@ -90,10 +90,7 @@ def accept(self, timeout: Optional[float] =3D 15.0) -> Q= MPMessage: self._aqmp.await_greeting =3D True self._aqmp.negotiate =3D True =20 - self._sync( - self._aqmp.start_server_and_accept(self._address), - timeout - ) + self._sync(self._aqmp.accept(), timeout) =20 ret =3D self._get_greeting() assert ret is not None --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691797883122.6152140170999; Mon, 7 Mar 2022 14:23:17 -0800 (PST) Received: from localhost ([::1]:39912 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLl7-0007vs-KC for importer@patchew.org; Mon, 07 Mar 2022 17:23:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48030) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdZ-0006Qt-8z for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:32925) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdV-00017U-Uj for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:28 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-433-U4F2Aak9PHmVOVMaJuAa4A-1; Mon, 07 Mar 2022 17:15:22 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2E7BE801AFC; Mon, 7 Mar 2022 22:15:21 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2AB6B5E253; Mon, 7 Mar 2022 22:15:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691325; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t8gz09ntCPRLWwaJkHCehmtew93vnxuUfAtDfOtlsXg=; b=dH0nnhIXOohJoE1Mo7mnaAyJlwlABWZcs3QdkWSMfrO5jTS88M7Cc9zr3JnYVpkBT8k5pR xKP2NDQHJ8AjD7orL80cy7wEW+kHfNLETT2ZyFv9AG+QTHHTKYvs3uL4nqzLj5QJTbksEQ exoYZnzHCAoMj6TNat2BTpGoiCE1Bcs= X-MC-Unique: U4F2Aak9PHmVOVMaJuAa4A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 10/11] python/aqmp: drop _bind_hack() Date: Mon, 7 Mar 2022 17:15:06 -0500 Message-Id: <20220307221507.1218892-11-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Beraldo Leal , qemu-block@nongnu.org, Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691799784100001 _bind_hack() was a quick fix to allow async QMP to call bind(2) prior to calling listen(2) and accept(2). This wasn't sufficient to fully address the race condition present in synchronous clients. With the race condition in legacy.py fixed (see the previous commit), there are no longer any users of _bind_hack(). Drop it. Fixes: b0b662bb2b3 Signed-off-by: John Snow Acked-by: Kevin Wolf Reviewed-by: Daniel P. Berrang=C3=A9 Message-id: 20220225205948.3693480-11-jsnow@redhat.com [Expanded commit message. --js] Signed-off-by: John Snow --- python/qemu/aqmp/legacy.py | 2 +- python/qemu/aqmp/protocol.py | 41 +++--------------------------------- 2 files changed, 4 insertions(+), 39 deletions(-) diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py index cb50e60564..46026e9fdc 100644 --- a/python/qemu/aqmp/legacy.py +++ b/python/qemu/aqmp/legacy.py @@ -57,7 +57,7 @@ def __init__(self, address: SocketAddrT, self._timeout: Optional[float] =3D None =20 if server: - self._sync(self._aqmp.start_server(address)) + self._sync(self._aqmp.start_server(self._address)) =20 _T =3D TypeVar('_T') =20 diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py index 2ecba14555..36fae57f27 100644 --- a/python/qemu/aqmp/protocol.py +++ b/python/qemu/aqmp/protocol.py @@ -18,7 +18,6 @@ from enum import Enum from functools import wraps import logging -import socket from ssl import SSLContext from typing import ( Any, @@ -242,9 +241,6 @@ def __init__(self, name: Optional[str] =3D None) -> Non= e: self._runstate =3D Runstate.IDLE self._runstate_changed: Optional[asyncio.Event] =3D None =20 - # Workaround for bind() - self._sock: Optional[socket.socket] =3D None - # Server state for start_server() and _incoming() self._server: Optional[asyncio.AbstractServer] =3D None self._accepted: Optional[asyncio.Event] =3D None @@ -535,34 +531,6 @@ async def _incoming(self, self._reader, self._writer =3D (reader, writer) self._accepted.set() =20 - def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None: - """ - Used to create a socket in advance of accept(). - - This is a workaround to ensure that we can guarantee timing of - precisely when a socket exists to avoid a connection attempt - bouncing off of nothing. - - Python 3.7+ adds a feature to separate the server creation and - listening phases instead, and should be used instead of this - hack. - """ - if isinstance(address, tuple): - family =3D socket.AF_INET - else: - family =3D socket.AF_UNIX - - sock =3D socket.socket(family, socket.SOCK_STREAM) - sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - - try: - sock.bind(address) - except: - sock.close() - raise - - self._sock =3D sock - @upper_half async def _do_start_server(self, address: SocketAddrT, ssl: Optional[SSLContext] =3D None) -> None: @@ -589,21 +557,19 @@ async def _do_start_server(self, address: SocketAddrT, if isinstance(address, tuple): coro =3D asyncio.start_server( self._incoming, - host=3DNone if self._sock else address[0], - port=3DNone if self._sock else address[1], + host=3Daddress[0], + port=3Daddress[1], ssl=3Dssl, backlog=3D1, limit=3Dself._limit, - sock=3Dself._sock, ) else: coro =3D asyncio.start_unix_server( self._incoming, - path=3DNone if self._sock else address, + path=3Daddress, ssl=3Dssl, backlog=3D1, limit=3Dself._limit, - sock=3Dself._sock, ) =20 # Allow runstate watchers to witness 'CONNECTING' state; some @@ -630,7 +596,6 @@ async def _do_accept(self) -> None: await self._accepted.wait() assert self._server is None self._accepted =3D None - self._sock =3D None =20 self.logger.debug("Connection accepted.") =20 --=20 2.34.1 From nobody Sat May 18 23:55:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646691871231594.7055256783525; Mon, 7 Mar 2022 14:24:31 -0800 (PST) Received: from localhost ([::1]:42030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRLmI-0000w2-7R for importer@patchew.org; Mon, 07 Mar 2022 17:24:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48054) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLda-0006U3-2u for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:57310) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRLdX-00018B-2P for qemu-devel@nongnu.org; Mon, 07 Mar 2022 17:15:29 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-253-3n7WIeQKOquvJKrBT6S49Q-1; Mon, 07 Mar 2022 17:15:23 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 54534800D55; Mon, 7 Mar 2022 22:15:22 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.10.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id 52A8845307; Mon, 7 Mar 2022 22:15:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646691326; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cGYvwJEOs9x4xXaD4X4n40vcQQquUaQRBJKh2J5D2bA=; b=ZKftHeNEt3gxMVGjGN+gzGFIGcuzCZOfsxI9Cs3FwWsU1DD7pT94pghyq4FJlM2br092Jc EklemFAeSsVj2xst0m1PkiCSIMTOjqXsRqDfXgh8R3dpoW9vG9wdkeZRx3gcMH4hHzBaIj HetL6HmW+5PgCM6Gg5DnQASWWvRNJ3A= X-MC-Unique: 3n7WIeQKOquvJKrBT6S49Q-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 11/11] scripts/qmp-shell-wrap: Fix import path Date: Mon, 7 Mar 2022 17:15:07 -0500 Message-Id: <20220307221507.1218892-12-jsnow@redhat.com> In-Reply-To: <20220307221507.1218892-1-jsnow@redhat.com> References: <20220307221507.1218892-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Beraldo Leal , qemu-block@nongnu.org, Kashyap Chamarthy , Markus Armbruster , Eduardo Habkost , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646691874493100001 Content-Type: text/plain; charset="utf-8" Mea culpa. Dan's patch wound up with the wrong import path because I re-ordered my most recent pull request and missed that this needed a fix on rebase. Fixes: 43912529 Reported-by: Kashyap Chamarthy Signed-off-by: John Snow Tested-by: Kashyap Chamarthy Message-id: 20220225170828.3418305-1-jsnow@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell-wrap | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qmp/qmp-shell-wrap b/scripts/qmp/qmp-shell-wrap index 9e94da114f..66846e36d1 100755 --- a/scripts/qmp/qmp-shell-wrap +++ b/scripts/qmp/qmp-shell-wrap @@ -4,7 +4,7 @@ import os import sys =20 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'pytho= n')) -from qemu.qmp import qmp_shell +from qemu.aqmp import qmp_shell =20 =20 if __name__ =3D=3D '__main__': --=20 2.34.1