From nobody Tue Sep 9 01:17:34 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756876664; cv=none; d=zohomail.com; s=zohoarc; b=VjLNN8h3GNxR6sw2yRwaEIAmsu5BibRW85FAKkGCo+neX51qS/8DssoNiZjv4exrjMaTIZltxXXiDj6Orw+K9o7XXuYO6NqkYllVRcLOS94vqKCg2VAY6KHGlUJ1UIYUliJq6Qrm31ZO6fYMjzSleosq0+i8RCA6mSn8Gct5UAs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756876664; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DMrZcYYh3K6amIjIAiIGx9VdKqeKnCNubcsS4x5vvUQ=; b=Vn0dm2y9wO2hKsjbe50a4Ht0FhnO+rLLAmalRlfPIfFTEBanksqXjwOsakHijqhEdNZVnr8yvZh62P5mGjpRJTHalRVzBphYkfw0uJZrv7yt/DAozsZUFiYTYYTL42kaFFoCPVqYaC+R/CRDY/arAwfoRDuOzzJ/1NxSJHtJDQs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756876664047193.39051251535966; Tue, 2 Sep 2025 22:17:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utfpH-0008UP-C4; Wed, 03 Sep 2025 01:14:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utfmz-0006KW-OB for qemu-devel@nongnu.org; Wed, 03 Sep 2025 01:12:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utfmu-0006ae-Rc for qemu-devel@nongnu.org; Wed, 03 Sep 2025 01:12:07 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-15-MPnl8GT4O7iNyHLM2QLUUA-1; Wed, 03 Sep 2025 01:12:00 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E67911956096; Wed, 3 Sep 2025 05:11:59 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.88.53]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CD1C319560A2; Wed, 3 Sep 2025 05:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756876324; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DMrZcYYh3K6amIjIAiIGx9VdKqeKnCNubcsS4x5vvUQ=; b=eECGTiI60d9JEkFhG51hhdRhcK94RrIFD8llkN4rXVxMzNcHViXM5qbP6gwbJkCJnBMhRx JiCqsBTzEscfP922FXxNPjF0gUOfKBoTW6qhMeuTM53WH8AzLP7MF/vuGjxsjfZ4Z/3JpE Kb2YbCNuow9YzNhk0tFKE7nq3EJ5t0s= X-MC-Unique: MPnl8GT4O7iNyHLM2QLUUA-1 X-Mimecast-MFC-AGG-ID: MPnl8GT4O7iNyHLM2QLUUA_1756876320 From: John Snow To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Cleber Rosa , =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Hanna Reitz , John Snow , Kevin Wolf , "Richard W.M. Jones" Subject: [PATCH v2 12/18] python: backport 'avoid creating additional event loops per thread' Date: Wed, 3 Sep 2025 01:11:18 -0400 Message-ID: <20250903051125.3020805-13-jsnow@redhat.com> In-Reply-To: <20250903051125.3020805-1-jsnow@redhat.com> References: <20250903051125.3020805-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756876665081124100 This commit is two backports squashed into one to avoid regressions. python: *really* remove get_event_loop A prior commit, aa1ff990, switched away from using get_event_loop *by default*, but this is not good enough to avoid deprecation warnings as `asyncio.get_event_loop_policy().get_event_loop()` is *also* deprecated. Replace this mechanism with explicit calls to asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to match. python: avoid creating additional event loops per thread "Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated behavior altogether by calling new_event_loop() directly if there was no loop currently running, but this has the unfortunate side effect of potentially creating multiple event loops per thread if tests instantiate multiple QMP connections in a single thread. This behavior is apparently not well-defined and causes problems in some, but not all, combinations of Python interpreter version and platform environment. Partially revert to Daniel Berrange's original patch, which calls get_event_loop and simply suppresses the deprecation warning in Python<=3D3.13. This time, however, additionally register new loops created with new_event_loop() so that future calls to get_event_loop() will return the loop already created. Reported-by: Richard W.M. Jones Reported-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow cherry picked from commit 21ce2ee4f2df87efe84a27b9c5112487f4670622 cherry picked from commit c08fb82b38212956ccffc03fc6d015c3979f42fe Signed-off-by: John Snow Reviewed-by: Daniel P. Berrang=C3=A9 --- python/qemu/qmp/legacy.py | 46 +++++++++++++++++++++++--------------- python/qemu/qmp/qmp_tui.py | 10 ++------- python/qemu/qmp/util.py | 27 ++++++++++++++++++++++ 3 files changed, 57 insertions(+), 26 deletions(-) diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index 735d42971e9..e46695ae2c8 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -38,6 +38,7 @@ from .error import QMPError from .protocol import Runstate, SocketAddrT from .qmp_client import QMPClient +from .util import get_or_create_event_loop =20 =20 #: QMPMessage is an entire QMP message of any kind. @@ -86,17 +87,13 @@ def __init__(self, "server argument should be False when passing a socket") =20 self._qmp =3D QMPClient(nickname) - - try: - self._aloop =3D asyncio.get_running_loop() - except RuntimeError: - # No running loop; since this is a sync shim likely to be - # used in fully sync programs, create one if neccessary. - self._aloop =3D asyncio.get_event_loop_policy().get_event_loop= () - self._address =3D address self._timeout: Optional[float] =3D None =20 + # This is a sync shim intended for use in fully synchronous + # programs. Create and set an event loop if necessary. + self._aloop =3D get_or_create_event_loop() + if server: assert not isinstance(self._address, socket.socket) self._sync(self._qmp.start_server(self._address)) @@ -313,17 +310,30 @@ def send_fd_scm(self, fd: int) -> None: self._qmp.send_fd_scm(fd) =20 def __del__(self) -> None: - if self._qmp.runstate =3D=3D Runstate.IDLE: - return + if self._qmp.runstate !=3D Runstate.IDLE: + self._qmp.logger.warning( + "QEMUMonitorProtocol object garbage collected without a pr= ior " + "call to close()" + ) =20 if not self._aloop.is_running(): - self.close() - else: - # Garbage collection ran while the event loop was running. - # Nothing we can do about it now, but if we don't raise our - # own error, the user will be treated to a lot of traceback - # they might not understand. + if self._qmp.runstate !=3D Runstate.IDLE: + # If the user neglected to close the QMP session and we + # are not currently running in an asyncio context, we + # have the opportunity to close the QMP session. If we + # do not do this, the error messages presented over + # dangling async resources may not make any sense to the + # user. + self.close() + + if self._qmp.runstate !=3D Runstate.IDLE: + # If QMP is still not quiesced, it means that the garbage + # collector ran from a context within the event loop and we + # are simply too late to take any corrective action. Raise + # our own error to give meaningful feedback to the user in + # order to prevent pages of asyncio stacktrace jargon. raise QMPError( - "QEMUMonitorProtocol.close()" - " was not called before object was garbage collected" + "QEMUMonitorProtocol.close() was not called before object = was " + "garbage collected, and could not be closed due to GC runn= ing " + "in the event loop" ) diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index 12bdc17c99e..d946c205131 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -51,7 +51,7 @@ from .message import DeserializationError, Message, UnexpectedTypeError from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient -from .util import pretty_traceback +from .util import get_or_create_event_loop, pretty_traceback =20 =20 # The name of the signal that is used to update the history list @@ -387,13 +387,7 @@ def run(self, debug: bool =3D False) -> None: """ screen =3D urwid.raw_display.Screen() screen.set_terminal_properties(256) - - try: - self.aloop =3D asyncio.get_running_loop() - except RuntimeError: - # No running asyncio event loop. Create one if necessary. - self.aloop =3D asyncio.get_event_loop_policy().get_event_loop() - + self.aloop =3D get_or_create_event_loop() self.aloop.set_debug(debug) =20 # Gracefully handle SIGTERM and SIGINT signals diff --git a/python/qemu/qmp/util.py b/python/qemu/qmp/util.py index 0b3e781373d..47ec39a8b5e 100644 --- a/python/qemu/qmp/util.py +++ b/python/qemu/qmp/util.py @@ -10,6 +10,7 @@ import sys import traceback from typing import TypeVar, cast +import warnings =20 =20 T =3D TypeVar('T') @@ -20,6 +21,32 @@ # -------------------------- =20 =20 +def get_or_create_event_loop() -> asyncio.AbstractEventLoop: + """ + Return this thread's current event loop, or create a new one. + + This function behaves similarly to asyncio.get_event_loop() in + Python<=3D3.13, where if there is no event loop currently associated + with the current context, it will create and register one. It should + generally not be used in any asyncio-native applications. + """ + try: + with warnings.catch_warnings(): + # Python <=3D 3.13 will trigger deprecation warnings if no + # event loop is set, but will create and set a new loop. + warnings.simplefilter("ignore") + loop =3D asyncio.get_event_loop() + except RuntimeError: + # Python 3.14+: No event loop set for this thread, + # create and set one. + loop =3D asyncio.new_event_loop() + # Set this loop as the current thread's loop, to be returned + # by calls to get_event_loop() in the future. + asyncio.set_event_loop(loop) + + return loop + + async def flush(writer: asyncio.StreamWriter) -> None: """ Utility function to ensure a StreamWriter is *fully* drained. --=20 2.50.1