From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163527115205218.32089818478221; Tue, 26 Oct 2021 10:59:12 -0700 (PDT) Received: from localhost ([::1]:39590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQj8-0006C2-FY for importer@patchew.org; Tue, 26 Oct 2021 13:59:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51380) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgS-0003Yb-3y for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46175) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgN-0004XV-QO for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:22 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-291-316eD2SvPbmyIpKSs5SM1A-1; Tue, 26 Oct 2021 13:56:15 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 96B2956C22; Tue, 26 Oct 2021 17:56:14 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAD135DF21; Tue, 26 Oct 2021 17:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UAVrEwDzEg01rdEaU0TH40MGZEH0sGEHGDA8ZQp1bM8=; b=QMrhhoOy8GIxJjMmVrhWfznoKyBkaisGiQ79FnekzROzpd6HYqBI14Q1TOONoYY1KmvmmR YA5WYkzTY+KCly6IDyHfbcDNQPyN7q8LpCak/TwwGiCyvADIOEPhloNiRtYJXaRomLyT6y xhPLKoJfFsJawOeTwbZ0x+wz1zBEwYU= X-MC-Unique: 316eD2SvPbmyIpKSs5SM1A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 1/8] python/machine: remove has_quit argument Date: Tue, 26 Oct 2021 13:56:05 -0400 Message-Id: <20211026175612.4127598-2-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271153450100001 Content-Type: text/plain; charset="utf-8" If we spy on the QMP commands instead, we don't need callers to remember to pass it. Seems like a fair trade-off. The one slightly weird bit is overloading this instance variable for wait(), where we use it to mean "don't issue the qmp 'quit' command". This means that wait() will "fail" if the QEMU process does not terminate of its own accord. In most cases, we probably did already actually issue quit -- some iotests do this -- but in some others, we may be waiting for QEMU to terminate for some other reason, such as a test wherein we tell the guest (directly) to shut down. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf --- python/qemu/machine/machine.py | 34 +++++++++++++++++++--------------- tests/qemu-iotests/040 | 7 +------ tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/255 | 2 +- 4 files changed, 22 insertions(+), 23 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 056d340e355..0bd40bc2f76 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -170,6 +170,7 @@ def __init__(self, self._console_socket: Optional[socket.socket] =3D None self._remove_files: List[str] =3D [] self._user_killed =3D False + self._quit_issued =3D False =20 def __enter__(self: _T) -> _T: return self @@ -368,6 +369,7 @@ def _post_shutdown(self) -> None: command =3D '' LOG.warning(msg, -int(exitcode), command) =20 + self._quit_issued =3D False self._user_killed =3D False self._launched =3D False =20 @@ -443,15 +445,13 @@ def _hard_shutdown(self) -> None: self._subp.kill() self._subp.wait(timeout=3D60) =20 - def _soft_shutdown(self, timeout: Optional[int], - has_quit: bool =3D False) -> None: + def _soft_shutdown(self, timeout: Optional[int]) -> None: """ Perform early cleanup, attempt to gracefully shut down the VM, and= wait for it to terminate. =20 :param timeout: Timeout in seconds for graceful shutdown. A value of None is an infinite wait. - :param has_quit: When True, don't attempt to issue 'quit' QMP comm= and =20 :raise ConnectionReset: On QMP communication errors :raise subprocess.TimeoutExpired: When timeout is exceeded waiting= for @@ -460,21 +460,19 @@ def _soft_shutdown(self, timeout: Optional[int], self._early_cleanup() =20 if self._qmp_connection: - if not has_quit: + if not self._quit_issued: # Might raise ConnectionReset - self._qmp.cmd('quit') + self.qmp('quit') =20 # May raise subprocess.TimeoutExpired self._subp.wait(timeout=3Dtimeout) =20 - def _do_shutdown(self, timeout: Optional[int], - has_quit: bool =3D False) -> None: + def _do_shutdown(self, timeout: Optional[int]) -> None: """ Attempt to shutdown the VM gracefully; fallback to a hard shutdown. =20 :param timeout: Timeout in seconds for graceful shutdown. A value of None is an infinite wait. - :param has_quit: When True, don't attempt to issue 'quit' QMP comm= and =20 :raise AbnormalShutdown: When the VM could not be shut down gracef= ully. The inner exception will likely be ConnectionReset or @@ -482,13 +480,13 @@ def _do_shutdown(self, timeout: Optional[int], may result in its own exceptions, likely subprocess.TimeoutExp= ired. """ try: - self._soft_shutdown(timeout, has_quit) + self._soft_shutdown(timeout) except Exception as exc: self._hard_shutdown() raise AbnormalShutdown("Could not perform graceful shutdown") \ from exc =20 - def shutdown(self, has_quit: bool =3D False, + def shutdown(self, hard: bool =3D False, timeout: Optional[int] =3D 30) -> None: """ @@ -498,7 +496,6 @@ def shutdown(self, has_quit: bool =3D False, If the VM has not yet been launched, or shutdown(), wait(), or kil= l() have already been called, this method does nothing. =20 - :param has_quit: When true, do not attempt to issue 'quit' QMP com= mand. :param hard: When true, do not attempt graceful shutdown, and suppress the SIGKILL warning log message. :param timeout: Optional timeout in seconds for graceful shutdown. @@ -512,7 +509,7 @@ def shutdown(self, has_quit: bool =3D False, self._user_killed =3D True self._hard_shutdown() else: - self._do_shutdown(timeout, has_quit) + self._do_shutdown(timeout) finally: self._post_shutdown() =20 @@ -529,7 +526,8 @@ def wait(self, timeout: Optional[int] =3D 30) -> None: :param timeout: Optional timeout in seconds. Default 30 seconds. A value of `None` is an infinite wait. """ - self.shutdown(has_quit=3DTrue, timeout=3Dtimeout) + self._quit_issued =3D True + self.shutdown(timeout=3Dtimeout) =20 def set_qmp_monitor(self, enabled: bool =3D True) -> None: """ @@ -574,7 +572,10 @@ def qmp(self, cmd: str, conv_keys =3D True =20 qmp_args =3D self._qmp_args(conv_keys, args) - return self._qmp.cmd(cmd, args=3Dqmp_args) + ret =3D self._qmp.cmd(cmd, args=3Dqmp_args) + if cmd =3D=3D 'quit' and 'error' not in ret and 'return' in ret: + self._quit_issued =3D True + return ret =20 def command(self, cmd: str, conv_keys: bool =3D True, @@ -585,7 +586,10 @@ def command(self, cmd: str, On failure raise an exception. """ qmp_args =3D self._qmp_args(conv_keys, args) - return self._qmp.command(cmd, **qmp_args) + ret =3D self._qmp.command(cmd, **qmp_args) + if cmd =3D=3D 'quit': + self._quit_issued =3D True + return ret =20 def get_qmp_event(self, wait: bool =3D False) -> Optional[QMPMessage]: """ diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index f3677de9dfd..6af5ab9e764 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -92,10 +92,9 @@ class TestSingleDrive(ImageCommitTestCase): self.vm.add_device('virtio-scsi') self.vm.add_device("scsi-hd,id=3Dscsi0,drive=3Ddrive0") self.vm.launch() - self.has_quit =3D False =20 def tearDown(self): - self.vm.shutdown(has_quit=3Dself.has_quit) + self.vm.shutdown() os.remove(test_img) os.remove(mid_img) os.remove(backing_img) @@ -127,8 +126,6 @@ class TestSingleDrive(ImageCommitTestCase): result =3D self.vm.qmp('quit') self.assert_qmp(result, 'return', {}) =20 - self.has_quit =3D True - # Same as above, but this time we add the filter after starting the job @iotests.skip_if_unsupported(['throttle']) def test_commit_plus_filter_and_quit(self): @@ -147,8 +144,6 @@ class TestSingleDrive(ImageCommitTestCase): result =3D self.vm.qmp('quit') self.assert_qmp(result, 'return', {}) =20 - self.has_quit =3D True - def test_device_not_found(self): result =3D self.vm.qmp('block-commit', device=3D'nonexistent', top= =3D'%s' % mid_img) self.assert_qmp(result, 'error/class', 'DeviceNotFound') diff --git a/tests/qemu-iotests/218 b/tests/qemu-iotests/218 index 325d8244fb9..4922b4d3b6f 100755 --- a/tests/qemu-iotests/218 +++ b/tests/qemu-iotests/218 @@ -187,4 +187,4 @@ with iotests.VM() as vm, \ log(vm.qmp('quit')) =20 with iotests.Timeout(5, 'Timeout waiting for VM to quit'): - vm.shutdown(has_quit=3DTrue) + vm.shutdown() diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255 index c43aa9c67ac..3d6d0e80cb5 100755 --- a/tests/qemu-iotests/255 +++ b/tests/qemu-iotests/255 @@ -123,4 +123,4 @@ with iotests.FilePath('src.qcow2') as src_path, \ vm.qmp_log('block-job-cancel', device=3D'job0') vm.qmp_log('quit') =20 - vm.shutdown(has_quit=3DTrue) + vm.shutdown() --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16352713386451019.3105593710968; Tue, 26 Oct 2021 11:02:18 -0700 (PDT) Received: from localhost ([::1]:45268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQm7-0001hb-HI for importer@patchew.org; Tue, 26 Oct 2021 14:02:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgS-0003Zg-VA for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52893) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgO-0004Xl-2K for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:24 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-18-L-0dErxBP3q9ckrREMJLHA-1; Tue, 26 Oct 2021 13:56:16 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 87D6110247A7; Tue, 26 Oct 2021 17:56:15 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBF805DF21; Tue, 26 Oct 2021 17:56:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aNr6SNMc0oBTtifDVANAHZqAY+Pl5cwKRo2yK0XeY90=; b=KE9o5TCsMiGXgNCU7boZP5SrdrwgPksB3Z4W5yg1N4TqoCrROU9QA7L4/BEorLHBxaXMne JeW45JexcGtfS+Wq4bN4wdoTAc7NawtFz7l1YfB/CMA9xnjUmnlZW8lvCG1LbIkCu5y/Da xMyy8oj4XilYChXk4/vP3LqkE+Me6jc= X-MC-Unique: L-0dErxBP3q9ckrREMJLHA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 2/8] python/machine: Handle QMP errors on close more meticulously Date: Tue, 26 Oct 2021 13:56:06 -0400 Message-Id: <20211026175612.4127598-3-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271343219100001 Content-Type: text/plain; charset="utf-8" To use the AQMP backend, Machine just needs to be a little more diligent about what happens when closing a QMP connection. The operation is no longer a freebie in the async world; it may return errors encountered in the async bottom half on incoming message receipt, etc. (AQMP's disconnect, ultimately, serves as the quiescence point where all async contexts are gathered together, and any final errors reported at that point.) Because async QMP continues to check for messages asynchronously, it's almost certainly likely that the loop will have exited due to EOF after issuing the last 'quit' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it -- if it was expected. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf --- python/qemu/machine/machine.py | 48 +++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 6 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 0bd40bc2f76..a0cf69786b4 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -342,9 +342,15 @@ def _post_shutdown(self) -> None: # Comprehensive reset for the failed launch case: self._early_cleanup() =20 - if self._qmp_connection: - self._qmp.close() - self._qmp_connection =3D None + try: + self._close_qmp_connection() + except Exception as err: # pylint: disable=3Dbroad-except + LOG.warning( + "Exception closing QMP connection: %s", + str(err) if str(err) else type(err).__name__ + ) + finally: + assert self._qmp_connection is None =20 self._close_qemu_log_file() =20 @@ -420,6 +426,31 @@ def _launch(self) -> None: close_fds=3DFalse) self._post_launch() =20 + def _close_qmp_connection(self) -> None: + """ + Close the underlying QMP connection, if any. + + Dutifully report errors that occurred while closing, but assume + that any error encountered indicates an abnormal termination + process and not a failure to close. + """ + if self._qmp_connection is None: + return + + try: + self._qmp.close() + except EOFError: + # EOF can occur as an Exception here when using the Async + # QMP backend. It indicates that the server closed the + # stream. If we successfully issued 'quit' at any point, + # then this was expected. If the remote went away without + # our permission, it's worth reporting that as an abnormal + # shutdown case. + if not (self._user_killed or self._quit_issued): + raise + finally: + self._qmp_connection =3D None + def _early_cleanup(self) -> None: """ Perform any cleanup that needs to happen before the VM exits. @@ -460,9 +491,14 @@ def _soft_shutdown(self, timeout: Optional[int]) -> No= ne: self._early_cleanup() =20 if self._qmp_connection: - if not self._quit_issued: - # Might raise ConnectionReset - self.qmp('quit') + try: + if not self._quit_issued: + # May raise ExecInterruptedError or StateError if the + # connection dies or has *already* died. + self.qmp('quit') + finally: + # Regardless, we want to quiesce the connection. + self._close_qmp_connection() =20 # May raise subprocess.TimeoutExpired self._subp.wait(timeout=3Dtimeout) --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635271180596745.3491031107686; Tue, 26 Oct 2021 10:59:40 -0700 (PDT) Received: from localhost ([::1]:40876 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQjb-00074Q-GK for importer@patchew.org; Tue, 26 Oct 2021 13:59:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgT-0003Zj-Ai for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:22005) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgO-0004XS-7j for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:25 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-399-e_diy8T8N_2MWpi_HhQbFQ-1; Tue, 26 Oct 2021 13:56:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 73AEA56C22; Tue, 26 Oct 2021 17:56:16 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA1285DF21; Tue, 26 Oct 2021 17:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCV2NWLWjTj4/oY1rx6+76Vvo3Z8mLGw0Qt3fdF9yLQ=; b=KkiBIOHlf01X4+IcYrtBDPbm2TK0eXJPNBba+K5K5/U5LR8RIitHXlz9O+SO+RU0kwm6S+ e2wyOpoQMlLumDrECp6rUbbltlwWTlvhqb2WJWM+lk3mdZJOyyY/j2gfP8Tb1vOorPYHkv jv1kH3Zzxyf/O9RXx63hJ7xNnACFp9k= X-MC-Unique: e_diy8T8N_2MWpi_HhQbFQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 3/8] python/aqmp: Remove scary message Date: Tue, 26 Oct 2021 13:56:07 -0400 Message-Id: <20211026175612.4127598-4-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271182048100001 Content-Type: text/plain; charset="utf-8" The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow Acked-by: Hanna Reitz Reviewed-by: Kevin Wolf --- python/qemu/aqmp/__init__.py | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py index d1b0e4dc3d3..880d5b6fa7f 100644 --- a/python/qemu/aqmp/__init__.py +++ b/python/qemu/aqmp/__init__.py @@ -22,7 +22,6 @@ # the COPYING file in the top-level directory. =20 import logging -import warnings =20 from .error import AQMPError from .events import EventListener @@ -31,17 +30,6 @@ from .qmp_client import ExecInterruptedError, ExecuteError, QMPClient =20 =20 -_WMSG =3D """ - -The Asynchronous QMP library is currently in development and its API -should be considered highly fluid and subject to change. It should -not be used by any other scripts checked into the QEMU tree. - -Proceed with caution! -""" - -warnings.warn(_WMSG, FutureWarning) - # Suppress logging unless an application engages it. logging.getLogger('qemu.aqmp').addHandler(logging.NullHandler()) =20 --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635271720115635.618620921187; Tue, 26 Oct 2021 11:08:40 -0700 (PDT) Received: from localhost ([::1]:55972 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQsH-0000bC-UF for importer@patchew.org; Tue, 26 Oct 2021 14:08:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51482) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgV-0003bc-MB for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39958) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgQ-0004Xr-NU for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-598-96bNrN9iNMyua-28iXVCjA-1; Tue, 26 Oct 2021 13:56:18 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D9BC802B7A; Tue, 26 Oct 2021 17:56:17 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF5675DF21; Tue, 26 Oct 2021 17:56:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VujbkLlEdDLV66q1zgixgLhSMurEoZPivssICNtAi8w=; b=Os8RE4buYgswmjC/aS+iTdQtXcL9wfVHdhiLXQBKk/vBZrYGG1cAbBWD+MvSSPaZjUXHm+ MKZRMoXUQF45elBOQ30AyGfrsM0BdiYS3jbAw1en7YG3Z1FGx32aj39LFLUFAH3oxSl0Ih ENUnGALkVVUa7Wqc4LZghXk2F80IwOc= X-MC-Unique: 96bNrN9iNMyua-28iXVCjA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 4/8] iotests: Accommodate async QMP Exception classes Date: Tue, 26 Oct 2021 13:56:08 -0400 Message-Id: <20211026175612.4127598-5-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271721344100001 Content-Type: text/plain; charset="utf-8" (But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf --- scripts/simplebench/bench_block_job.py | 3 ++- tests/qemu-iotests/tests/mirror-top-perms | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/simplebench/bench_block_job.py b/scripts/simplebench/b= ench_block_job.py index 4f03c121697..a403c35b08f 100755 --- a/scripts/simplebench/bench_block_job.py +++ b/scripts/simplebench/bench_block_job.py @@ -28,6 +28,7 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'pytho= n')) from qemu.machine import QEMUMachine from qemu.qmp import QMPConnectError +from qemu.aqmp import ConnectError =20 =20 def bench_block_job(cmd, cmd_args, qemu_args): @@ -49,7 +50,7 @@ def bench_block_job(cmd, cmd_args, qemu_args): vm.launch() except OSError as e: return {'error': 'popen failed: ' + str(e)} - except (QMPConnectError, socket.timeout): + except (QMPConnectError, ConnectError, socket.timeout): return {'error': 'qemu failed: ' + str(vm.get_log())} =20 try: diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index 3d475aa3a54..a2d5c269d7a 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -21,8 +21,9 @@ =20 import os =20 -from qemu import qmp +from qemu.aqmp import ConnectError from qemu.machine import machine +from qemu.qmp import QMPConnectError =20 import iotests from iotests import qemu_img @@ -102,7 +103,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.launch() print('ERROR: VM B launched successfully, this should not have= ' 'happened') - except qmp.QMPConnectError: + except (QMPConnectError, ConnectError): assert 'Is another process using the image' in self.vm_b.get_l= og() =20 result =3D self.vm.qmp('block-job-cancel', --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635271571229832.53358267651; Tue, 26 Oct 2021 11:06:11 -0700 (PDT) Received: from localhost ([::1]:50940 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQpu-0005Wz-7E for importer@patchew.org; Tue, 26 Oct 2021 14:06:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgW-0003be-7G for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:26727) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgS-0004YK-AS for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-566-o-nRK4dAOIyjmfqne2QtGQ-1; Tue, 26 Oct 2021 13:56:19 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6DA87801FCE; Tue, 26 Oct 2021 17:56:18 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id A12405DF21; Tue, 26 Oct 2021 17:56:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6F7EokPgG8CupzwpVYA7pJZn03fGA9vq4abIY7XobU0=; b=OSPcCNgpT9BbAuBAF2PZLffv+Eo1fWU31qMK8z1hK4SxLr0aNaxUBstq4HQlgzkXm76oHz iT0wc5lsNtiUp6zZ3TwIXZ+ZjC1c4e9C7OpUNunZld9lN2zLEBDzxEdgDKOebcC/Vt5gZw PSF3OjO8L+CyVEOi07AbZCKYF22ReYw= X-MC-Unique: o-nRK4dAOIyjmfqne2QtGQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 5/8] iotests: Conditionally silence certain AQMP errors Date: Tue, 26 Oct 2021 13:56:09 -0400 Message-Id: <20211026175612.4127598-6-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271572380100001 Content-Type: text/plain; charset="utf-8" AQMP likes to be very chatty about errors it encounters. In general, this is good because it allows us to get good diagnostic information for otherwise complex async failures. For example, during a failed QMP connection attempt, we might see: +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Negotiation failed: EOFError +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Failed to establish session: EOFE= rror This might be nice in iotests output, because failure scenarios involving the new QMP library will be spelled out plainly in the output diffs. For tests that are intentionally causing this scenario though, filtering that log output could be a hassle. For now, add a context manager that simply lets us toggle this output off during a critical region. (Additionally, a forthcoming patch allows the use of either legacy or async QMP to be toggled with an environment variable. In this circumstance, we can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 20 +++++++++++++++++++- tests/qemu-iotests/tests/mirror-top-perms | 12 ++++++++---- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e5fff6ddcfc..e2f9d873ada 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -30,7 +30,7 @@ import subprocess import sys import time -from typing import (Any, Callable, Dict, Iterable, +from typing import (Any, Callable, Dict, Iterable, Iterator, List, Optional, Sequence, TextIO, Tuple, Type, TypeVar) import unittest =20 @@ -114,6 +114,24 @@ sample_img_dir =3D os.environ['SAMPLE_IMG_DIR'] =20 =20 +@contextmanager +def change_log_level( + logger_name: str, level: int =3D logging.CRITICAL) -> Iterator[Non= e]: + """ + Utility function for temporarily changing the log level of a logger. + + This can be used to silence errors that are expected or uninteresting. + """ + _logger =3D logging.getLogger(logger_name) + current_level =3D _logger.level + _logger.setLevel(level) + + try: + yield + finally: + _logger.setLevel(current_level) + + def unarchive_sample_image(sample, fname): sample_fname =3D os.path.join(sample_img_dir, sample + '.bz2') with bz2.open(sample_fname) as f_in, open(fname, 'wb') as f_out: diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index a2d5c269d7a..0a51a613f39 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -26,7 +26,7 @@ from qemu.machine import machine from qemu.qmp import QMPConnectError =20 import iotests -from iotests import qemu_img +from iotests import change_log_level, qemu_img =20 =20 image_size =3D 1 * 1024 * 1024 @@ -100,9 +100,13 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.add_blockdev(f'file,node-name=3Ddrive0,filename=3D{sourc= e}') self.vm_b.add_device('virtio-blk,drive=3Ddrive0,share-rw=3Don') try: - self.vm_b.launch() - print('ERROR: VM B launched successfully, this should not have= ' - 'happened') + # Silence AQMP errors temporarily. + # TODO: Remove this and just allow the errors to be logged when + # AQMP fully replaces QMP. + with change_log_level('qemu.aqmp'): + self.vm_b.launch() + print('ERROR: VM B launched successfully, ' + 'this should not have happened') except (QMPConnectError, ConnectError): assert 'Is another process using the image' in self.vm_b.get_l= og() =20 --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635272003030190.73935130629422; Tue, 26 Oct 2021 11:13:23 -0700 (PDT) Received: from localhost ([::1]:35814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQws-00064N-2s for importer@patchew.org; Tue, 26 Oct 2021 14:13:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgY-0003cp-1N for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23121) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgS-0004Yd-Nm for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:29 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-142-YeiO3I32Mbyp8r4eDCsUPg-1; Tue, 26 Oct 2021 13:56:20 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 741AD10247A7; Tue, 26 Oct 2021 17:56:19 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8FB015DF21; Tue, 26 Oct 2021 17:56:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zg98vKT4BEF48sXoFpc39NSWEIJX99bedc0aLM9zakk=; b=WJDaACijmRjI1WBUH/3RlI7iyf4W7SmPthyJvJWcq1aY0fV4zfCINgZwP7m2XSIpvYkHPD MVU/eyPZoJ/OHTV/98BBaA8RQY2FtiQas7kAUnBDd2pEK6PQqKq2P3NxBkvZedWpQExMvX KUvBABMO45/5+d/2XOg78lD3TqglcQ0= X-MC-Unique: YeiO3I32Mbyp8r4eDCsUPg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 6/8] iotests/300: avoid abnormal shutdown race condition Date: Tue, 26 Oct 2021 13:56:10 -0400 Message-Id: <20211026175612.4127598-7-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635272004815100005 Content-Type: text/plain; charset="utf-8" Wait for the destination VM to close itself instead of racing to shut it down first, which produces different error log messages from AQMP depending on precisely when we tried to shut it down. (For example: We may try to issue 'quit' immediately prior to the target VM closing its QMP socket, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- tests/qemu-iotests/300 | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 index 10f9f2a8da6..dbd28384ec3 100755 --- a/tests/qemu-iotests/300 +++ b/tests/qemu-iotests/300 @@ -24,8 +24,6 @@ import random import re from typing import Dict, List, Optional =20 -from qemu.machine import machine - import iotests =20 =20 @@ -461,12 +459,11 @@ class TestBlockBitmapMappingErrors(TestDirtyBitmapMig= ration): f"'{self.src_node_name}': Name is longer than 255 by= tes", log) =20 - # Expect abnormal shutdown of the destination VM because of - # the failed migration - try: - self.vm_b.shutdown() - except machine.AbnormalShutdown: - pass + # Destination VM will terminate w/ error of its own accord + # due to the failed migration. + self.vm_b.wait() + rc =3D self.vm_b.exitcode() + assert rc is not None and rc > 0 =20 def test_aliased_bitmap_name_too_long(self) -> None: # Longer than the maximum for bitmap names --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635271478209587.2519655459657; Tue, 26 Oct 2021 11:04:38 -0700 (PDT) Received: from localhost ([::1]:49464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQoN-0004Xf-Vp for importer@patchew.org; Tue, 26 Oct 2021 14:04:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51552) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQga-0003eA-Lw for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21039) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgT-0004Yu-7x for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-186-onZLZHGNOmKwWVnDnKlZgA-1; Tue, 26 Oct 2021 13:56:21 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9293681CBDB; Tue, 26 Oct 2021 17:56:20 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F521ADC8; Tue, 26 Oct 2021 17:56:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pNkumLhAQCxxnmqXVCE5nbKfqoIhJY7Inz+zB/lk0AE=; b=Te2LRuYI/R+cCdpu0F2Ts//UNzenKelKFEe1TbHMm/oXKGj9HcyAk+Rm9afAYedxDkdEa7 vvN6w65LryQN0W55xsQDFyzdgszkQZfZhKABdMs2v4tISnbziXbA1+21fKcMfiKFt65K0/ 4OFuxT83QBDlHoeQQrPpTjcwnGLuXvA= X-MC-Unique: onZLZHGNOmKwWVnDnKlZgA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 7/8] python/aqmp: Create sync QMP wrapper for iotests Date: Tue, 26 Oct 2021 13:56:11 -0400 Message-Id: <20211026175612.4127598-8-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271479443100001 Content-Type: text/plain; charset="utf-8" This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz Reviewed-by: Kevin Wolf --- python/qemu/aqmp/legacy.py | 138 +++++++++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100644 python/qemu/aqmp/legacy.py diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py new file mode 100644 index 00000000000..9e7b9fb80b9 --- /dev/null +++ b/python/qemu/aqmp/legacy.py @@ -0,0 +1,138 @@ +""" +Sync QMP Wrapper + +This class pretends to be qemu.qmp.QEMUMonitorProtocol. +""" + +import asyncio +from typing import ( + Awaitable, + List, + Optional, + TypeVar, + Union, +) + +import qemu.qmp +from qemu.qmp import QMPMessage, QMPReturnValue, SocketAddrT + +from .qmp_client import QMPClient + + +# pylint: disable=3Dmissing-docstring + + +class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol): + def __init__(self, address: SocketAddrT, + server: bool =3D False, + nickname: Optional[str] =3D None): + + # pylint: disable=3Dsuper-init-not-called + self._aqmp =3D QMPClient(nickname) + self._aloop =3D asyncio.get_event_loop() + self._address =3D address + self._timeout: Optional[float] =3D None + + _T =3D TypeVar('_T') + + def _sync( + self, future: Awaitable[_T], timeout: Optional[float] =3D None + ) -> _T: + return self._aloop.run_until_complete( + asyncio.wait_for(future, timeout=3Dtimeout) + ) + + def _get_greeting(self) -> Optional[QMPMessage]: + if self._aqmp.greeting is not None: + # pylint: disable=3Dprotected-access + return self._aqmp.greeting._asdict() + return None + + # __enter__ and __exit__ need no changes + # parse_address needs no changes + + def connect(self, negotiate: bool =3D True) -> Optional[QMPMessage]: + self._aqmp.await_greeting =3D negotiate + self._aqmp.negotiate =3D negotiate + + self._sync( + self._aqmp.connect(self._address) + ) + return self._get_greeting() + + def accept(self, timeout: Optional[float] =3D 15.0) -> QMPMessage: + self._aqmp.await_greeting =3D True + self._aqmp.negotiate =3D True + + self._sync( + self._aqmp.accept(self._address), + timeout + ) + + ret =3D self._get_greeting() + assert ret is not None + return ret + + def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: + return dict( + self._sync( + # pylint: disable=3Dprotected-access + + # _raw() isn't a public API, because turning off + # automatic ID assignment is discouraged. For + # compatibility with iotests *only*, do it anyway. + self._aqmp._raw(qmp_cmd, assign_id=3DFalse), + self._timeout + ) + ) + + # Default impl of cmd() delegates to cmd_obj + + def command(self, cmd: str, **kwds: object) -> QMPReturnValue: + return self._sync( + self._aqmp.execute(cmd, kwds), + self._timeout + ) + + def pull_event(self, + wait: Union[bool, float] =3D False) -> Optional[QMPMess= age]: + if not wait: + # wait is False/0: "do not wait, do not except." + if self._aqmp.events.empty(): + return None + + # If wait is 'True', wait forever. If wait is False/0, the events + # queue must not be empty; but it still needs some real amount + # of time to complete. + timeout =3D None + if wait and isinstance(wait, float): + timeout =3D wait + + return dict( + self._sync( + self._aqmp.events.get(), + timeout + ) + ) + + def get_events(self, wait: Union[bool, float] =3D False) -> List[QMPMe= ssage]: + events =3D [dict(x) for x in self._aqmp.events.clear()] + if events: + return events + + event =3D self.pull_event(wait) + return [event] if event is not None else [] + + def clear_events(self) -> None: + self._aqmp.events.clear() + + def close(self) -> None: + self._sync( + self._aqmp.disconnect() + ) + + def settimeout(self, timeout: Optional[float]) -> None: + self._timeout =3D timeout + + def send_fd_scm(self, fd: int) -> None: + self._aqmp.send_fd_scm(fd) --=20 2.31.1 From nobody Sun May 12 05:13:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163527161160367.87359032583515; Tue, 26 Oct 2021 11:06:51 -0700 (PDT) Received: from localhost ([::1]:52844 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mfQqY-0006oB-Gk for importer@patchew.org; Tue, 26 Oct 2021 14:06:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgX-0003cf-67 for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:55981) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfQgS-0004Ya-Mx for qemu-devel@nongnu.org; Tue, 26 Oct 2021 13:56:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-429-yxULgMXINuGz4Eb3Flw9IQ-1; Tue, 26 Oct 2021 13:56:22 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 806951023F4D; Tue, 26 Oct 2021 17:56:21 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.51]) by smtp.corp.redhat.com (Postfix) with ESMTP id B41475DF21; Tue, 26 Oct 2021 17:56:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635270983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7bs1uEySQ3f1s/c22vKBVNLQv/ngA339gcy2wgPm8eE=; b=P0tyv1Li66onn4NTLOnWSUb8KoZ1fIOdPxIX15HbWUTMdRj7VthEzzD0abUxZqNxYNTxKo JmWe+uBmVcdyf+LHJ3k5z9as9Q0XN5LYno4kCnye6R8V0AkbnbNDJpVxtSWIeaH5Z0Lz53 7HykbBB8Akp8PIW8jOonpJirIpEgtS4= X-MC-Unique: yxULgMXINuGz4Eb3Flw9IQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 8/8] python, iotests: replace qmp with aqmp Date: Tue, 26 Oct 2021 13:56:12 -0400 Message-Id: <20211026175612.4127598-9-jsnow@redhat.com> In-Reply-To: <20211026175612.4127598-1-jsnow@redhat.com> References: <20211026175612.4127598-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635271612243100001 Content-Type: text/plain; charset="utf-8" Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync wrapper from qemu.aqmp instead. Add an escape hatch in the form of the environment variable QEMU_PYTHON_LEGACY_QMP which allows you to cajole QEMUMachine into using the old implementation, proving that both implementations work concurrently. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- python/qemu/machine/machine.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index a0cf69786b4..a487c397459 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -41,7 +41,6 @@ ) =20 from qemu.qmp import ( # pylint: disable=3Dimport-error - QEMUMonitorProtocol, QMPMessage, QMPReturnValue, SocketAddrT, @@ -50,6 +49,12 @@ from . import console_socket =20 =20 +if os.environ.get('QEMU_PYTHON_LEGACY_QMP'): + from qemu.qmp import QEMUMonitorProtocol +else: + from qemu.aqmp.legacy import QEMUMonitorProtocol + + LOG =3D logging.getLogger(__name__) =20 =20 --=20 2.31.1