From nobody Wed Sep 10 05:57:23 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1757358692; cv=none; d=zohomail.com; s=zohoarc; b=AenUcmp7WjKQr6mYlMm38gFeUs//zAinHfgjVW0Lff7wyALDmbWjEwQAN4S+eK+AU+j28EVZg3wTD2waL2tqj6HLiIf8cPwvJzkWhpC+tlH63EvAXfaxuYrAQAc8TZ3MeygTXZR/k3IQayogv1+shWzKBAgh1kTFjVIiaKsQDKg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1757358692; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3BfWxpJbAnJ9d6T+UkvBd172fXR82WSe+8/48aB6pko=; b=n6Rqj4sswppX9eMIJ9Ll0oMGJH5kYql1gfogl9JSdbs/MFiBQV7SVOoa8nXbTDLbQSp6wiwHp5XqpNZSTuHuRJ9N+uMJopr/PJNZQc2ZAUv6iMPF1JhPrkSanMaeRBZ4SwMQtuWq4DNMjrZ6G5NZKXGSALLJNlbErH7VMmDtaoM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1757358692083533.1106034382192; Mon, 8 Sep 2025 12:11:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uvhAN-0006cd-3r; Mon, 08 Sep 2025 15:04:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uvhAH-0006ax-Q6 for qemu-devel@nongnu.org; Mon, 08 Sep 2025 15:04:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uvh9v-0007ya-Fj for qemu-devel@nongnu.org; Mon, 08 Sep 2025 15:04:33 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-232-cs67CJxrNPK1e2d0UESlbA-1; Mon, 08 Sep 2025 15:04:05 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5A38219560A2; Mon, 8 Sep 2025 19:04:04 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.88.36]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8DC4F1955F24; Mon, 8 Sep 2025 19:04:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1757358248; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3BfWxpJbAnJ9d6T+UkvBd172fXR82WSe+8/48aB6pko=; b=P2HRNj5nncbDEgKUDmIVX8ysylPrtbp2cXHZNrLVZ/lsWbj76W1A5Sgyh7HIiKyqfn+9YO cKHMSfh/Szwn7aSxeFI+S5fdSP12QukgiYm1BG6iAZ+UrrK+qu9A+YIZyOup8ZFC0AL39K 2/6ACZTOvEjo1cUJQLQ/fZAFsGgIl1g= X-MC-Unique: cs67CJxrNPK1e2d0UESlbA-1 X-Mimecast-MFC-AGG-ID: cs67CJxrNPK1e2d0UESlbA_1757358244 From: John Snow To: qemu-devel@nongnu.org Cc: Kevin Wolf , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Hanna Reitz , John Snow , Cleber Rosa , "Richard W.M. Jones" Subject: [PATCH v3 13/19] python: backport 'avoid creating additional event loops per thread' Date: Mon, 8 Sep 2025 15:03:12 -0400 Message-ID: <20250908190318.3331728-14-jsnow@redhat.com> In-Reply-To: <20250908190318.3331728-1-jsnow@redhat.com> References: <20250908190318.3331728-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1757358694830124100 This commit is two backports squashed into one to avoid regressions. python: *really* remove get_event_loop A prior commit, aa1ff990, switched away from using get_event_loop *by default*, but this is not good enough to avoid deprecation warnings as `asyncio.get_event_loop_policy().get_event_loop()` is *also* deprecated. Replace this mechanism with explicit calls to asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to match. python: avoid creating additional event loops per thread "Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated behavior altogether by calling new_event_loop() directly if there was no loop currently running, but this has the unfortunate side effect of potentially creating multiple event loops per thread if tests instantiate multiple QMP connections in a single thread. This behavior is apparently not well-defined and causes problems in some, but not all, combinations of Python interpreter version and platform environment. Partially revert to Daniel Berrange's original patch, which calls get_event_loop and simply suppresses the deprecation warning in Python<=3D3.13. This time, however, additionally register new loops created with new_event_loop() so that future calls to get_event_loop() will return the loop already created. Reported-by: Richard W.M. Jones Reported-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow cherry picked from commit python-qemu-qmp@21ce2ee4f2df87efe84a27b9c5112487f= 4670622 cherry picked from commit python-qemu-qmp@c08fb82b38212956ccffc03fc6d015c39= 79f42fe Signed-off-by: John Snow Reviewed-by: Daniel P. Berrang=C3=A9 --- python/qemu/qmp/legacy.py | 46 +++++++++++++++++++++++--------------- python/qemu/qmp/qmp_tui.py | 10 ++------- python/qemu/qmp/util.py | 27 ++++++++++++++++++++++ 3 files changed, 57 insertions(+), 26 deletions(-) diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index 735d42971e9..e46695ae2c8 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -38,6 +38,7 @@ from .error import QMPError from .protocol import Runstate, SocketAddrT from .qmp_client import QMPClient +from .util import get_or_create_event_loop =20 =20 #: QMPMessage is an entire QMP message of any kind. @@ -86,17 +87,13 @@ def __init__(self, "server argument should be False when passing a socket") =20 self._qmp =3D QMPClient(nickname) - - try: - self._aloop =3D asyncio.get_running_loop() - except RuntimeError: - # No running loop; since this is a sync shim likely to be - # used in fully sync programs, create one if neccessary. - self._aloop =3D asyncio.get_event_loop_policy().get_event_loop= () - self._address =3D address self._timeout: Optional[float] =3D None =20 + # This is a sync shim intended for use in fully synchronous + # programs. Create and set an event loop if necessary. + self._aloop =3D get_or_create_event_loop() + if server: assert not isinstance(self._address, socket.socket) self._sync(self._qmp.start_server(self._address)) @@ -313,17 +310,30 @@ def send_fd_scm(self, fd: int) -> None: self._qmp.send_fd_scm(fd) =20 def __del__(self) -> None: - if self._qmp.runstate =3D=3D Runstate.IDLE: - return + if self._qmp.runstate !=3D Runstate.IDLE: + self._qmp.logger.warning( + "QEMUMonitorProtocol object garbage collected without a pr= ior " + "call to close()" + ) =20 if not self._aloop.is_running(): - self.close() - else: - # Garbage collection ran while the event loop was running. - # Nothing we can do about it now, but if we don't raise our - # own error, the user will be treated to a lot of traceback - # they might not understand. + if self._qmp.runstate !=3D Runstate.IDLE: + # If the user neglected to close the QMP session and we + # are not currently running in an asyncio context, we + # have the opportunity to close the QMP session. If we + # do not do this, the error messages presented over + # dangling async resources may not make any sense to the + # user. + self.close() + + if self._qmp.runstate !=3D Runstate.IDLE: + # If QMP is still not quiesced, it means that the garbage + # collector ran from a context within the event loop and we + # are simply too late to take any corrective action. Raise + # our own error to give meaningful feedback to the user in + # order to prevent pages of asyncio stacktrace jargon. raise QMPError( - "QEMUMonitorProtocol.close()" - " was not called before object was garbage collected" + "QEMUMonitorProtocol.close() was not called before object = was " + "garbage collected, and could not be closed due to GC runn= ing " + "in the event loop" ) diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index 12bdc17c99e..d946c205131 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -51,7 +51,7 @@ from .message import DeserializationError, Message, UnexpectedTypeError from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient -from .util import pretty_traceback +from .util import get_or_create_event_loop, pretty_traceback =20 =20 # The name of the signal that is used to update the history list @@ -387,13 +387,7 @@ def run(self, debug: bool =3D False) -> None: """ screen =3D urwid.raw_display.Screen() screen.set_terminal_properties(256) - - try: - self.aloop =3D asyncio.get_running_loop() - except RuntimeError: - # No running asyncio event loop. Create one if necessary. - self.aloop =3D asyncio.get_event_loop_policy().get_event_loop() - + self.aloop =3D get_or_create_event_loop() self.aloop.set_debug(debug) =20 # Gracefully handle SIGTERM and SIGINT signals diff --git a/python/qemu/qmp/util.py b/python/qemu/qmp/util.py index 0b3e781373d..47ec39a8b5e 100644 --- a/python/qemu/qmp/util.py +++ b/python/qemu/qmp/util.py @@ -10,6 +10,7 @@ import sys import traceback from typing import TypeVar, cast +import warnings =20 =20 T =3D TypeVar('T') @@ -20,6 +21,32 @@ # -------------------------- =20 =20 +def get_or_create_event_loop() -> asyncio.AbstractEventLoop: + """ + Return this thread's current event loop, or create a new one. + + This function behaves similarly to asyncio.get_event_loop() in + Python<=3D3.13, where if there is no event loop currently associated + with the current context, it will create and register one. It should + generally not be used in any asyncio-native applications. + """ + try: + with warnings.catch_warnings(): + # Python <=3D 3.13 will trigger deprecation warnings if no + # event loop is set, but will create and set a new loop. + warnings.simplefilter("ignore") + loop =3D asyncio.get_event_loop() + except RuntimeError: + # Python 3.14+: No event loop set for this thread, + # create and set one. + loop =3D asyncio.new_event_loop() + # Set this loop as the current thread's loop, to be returned + # by calls to get_event_loop() in the future. + asyncio.set_event_loop(loop) + + return loop + + async def flush(writer: asyncio.StreamWriter) -> None: """ Utility function to ensure a StreamWriter is *fully* drained. --=20 2.50.1