From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758581; cv=none; d=zohomail.com; s=zohoarc; b=hzUdFI6At1jM4UbtLkqfyourbb+MtEShzt4vHUKRY9DVxJ4x2+Bff0IF07UkrAXiox+fHIjCnyf3X5y122oh0dyizBmj7zJtX3hFFKdHsNVWw2HxnAKF0+5TTk3vANbjjWsg+tUxBEwogG4QZ7Fw4TUPJNn/LrO7eGpwgjKo7R4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758581; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6RVpBkHk7NMO1lI1YYFOF4xP7A9ERIrBpsLNs7g8GsU=; b=BaCY1qPY+cdscdDY0bm3GtlfDxnu8YD87Wf2y5U8WEbaqxtGBE9GDUhji4S+gdL5MjEATsa1GdPHdbSSn7gY2b/uBi24isJrI5RdfpeLeQSGFzy6jyltzWJncXRH4whPXliS1CcV0xu5C+DD+lGVXGZ3AIfwn6ZSPBtGHcwpTAs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758581655949.5536420145155; Mon, 1 Sep 2025 13:29:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7Y-00046X-5g; Mon, 01 Sep 2025 16:27:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7T-00045B-ST for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7R-0008Ko-Sj for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:15 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-692-DpHa3maeOJi28sETwRtInQ-1; Mon, 01 Sep 2025 16:27:10 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3FE131800561; Mon, 1 Sep 2025 20:27:09 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CBDF119560B4; Mon, 1 Sep 2025 20:27:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758433; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6RVpBkHk7NMO1lI1YYFOF4xP7A9ERIrBpsLNs7g8GsU=; b=HhhtTaEkC3eFZG/K301iBjAIdlWzND/skS6dH8chIt1Y4oZl59q9FbH4uak5cA5lcqXXn1 klNqfuZeDFIClpC8ztGVA+e+O1eWyr9XhmMC/u4AMgSrvXiZZjf7+cnm11yliRn8detaQE sMeNCX2xBsVSI/wzpQ5NDX9GUdevi0c= X-MC-Unique: DpHa3maeOJi28sETwRtInQ-1 X-Mimecast-MFC-AGG-ID: DpHa3maeOJi28sETwRtInQ_1756758429 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 01/19] python: backport 'Change error classes to have better repr methods' Date: Mon, 1 Sep 2025 16:26:43 -0400 Message-ID: <20250901202702.2971212-2-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758583621124100 Content-Type: text/plain; charset="utf-8" By passing all of the arguments to the base class and overriding the __str__ method when we want a different "human readable" message that isn't just printing the list of arguments, we can ensure that all custom error classes have a reasonable __repr__ implementation. In the case of ExecuteError, the pseudo-field that isn't actually correlated to an input argument can be re-imagined as a read-only property; this forces consistency in the class and makes the repr output more obviously correct. Signed-off-by: John Snow cherry picked from commit afdb7893f3b34212da4259b7202973f9a8cb85b3 --- python/qemu/qmp/error.py | 7 +++++-- python/qemu/qmp/message.py | 12 ++++++------ python/qemu/qmp/protocol.py | 7 +++++-- python/qemu/qmp/qmp_client.py | 20 +++++++++++++------- 4 files changed, 29 insertions(+), 17 deletions(-) diff --git a/python/qemu/qmp/error.py b/python/qemu/qmp/error.py index 24ba4d50541..c87b078f620 100644 --- a/python/qemu/qmp/error.py +++ b/python/qemu/qmp/error.py @@ -44,7 +44,10 @@ class ProtocolError(QMPError): =20 :param error_message: Human-readable string describing the error. """ - def __init__(self, error_message: str): - super().__init__(error_message) + def __init__(self, error_message: str, *args: object): + super().__init__(error_message, *args) #: Human-readable error message, without any prefix. self.error_message: str =3D error_message + + def __str__(self) -> str: + return self.error_message diff --git a/python/qemu/qmp/message.py b/python/qemu/qmp/message.py index f76ccc90746..c2e9dd0dd54 100644 --- a/python/qemu/qmp/message.py +++ b/python/qemu/qmp/message.py @@ -178,15 +178,15 @@ class DeserializationError(ProtocolError): :param raw: The raw `bytes` that prompted the failure. """ def __init__(self, error_message: str, raw: bytes): - super().__init__(error_message) + super().__init__(error_message, raw) #: The raw `bytes` that were not understood as JSON. self.raw: bytes =3D raw =20 def __str__(self) -> str: - return "\n".join([ + return "\n".join(( super().__str__(), f" raw bytes were: {str(self.raw)}", - ]) + )) =20 =20 class UnexpectedTypeError(ProtocolError): @@ -197,13 +197,13 @@ class UnexpectedTypeError(ProtocolError): :param value: The deserialized JSON value that wasn't an object. """ def __init__(self, error_message: str, value: object): - super().__init__(error_message) + super().__init__(error_message, value) #: The JSON value that was expected to be an object. self.value: object =3D value =20 def __str__(self) -> str: strval =3D json.dumps(self.value, indent=3D2) - return "\n".join([ + return "\n".join(( super().__str__(), f" json value was: {strval}", - ]) + )) diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py index a4ffdfad51b..86e588881b7 100644 --- a/python/qemu/qmp/protocol.py +++ b/python/qemu/qmp/protocol.py @@ -80,7 +80,7 @@ class ConnectError(QMPError): :param exc: The root-cause exception. """ def __init__(self, error_message: str, exc: Exception): - super().__init__(error_message) + super().__init__(error_message, exc) #: Human-readable error string self.error_message: str =3D error_message #: Wrapped root cause exception @@ -108,11 +108,14 @@ class StateError(QMPError): """ def __init__(self, error_message: str, state: Runstate, required: Runstate): - super().__init__(error_message) + super().__init__(error_message, state, required) self.error_message =3D error_message self.state =3D state self.required =3D required =20 + def __str__(self) -> str: + return self.error_message + =20 F =3D TypeVar('F', bound=3DCallable[..., Any]) # pylint: disable=3Dinvali= d-name =20 diff --git a/python/qemu/qmp/qmp_client.py b/python/qemu/qmp/qmp_client.py index 2a817f9db33..a87fb565ab5 100644 --- a/python/qemu/qmp/qmp_client.py +++ b/python/qemu/qmp/qmp_client.py @@ -41,7 +41,7 @@ class _WrappedProtocolError(ProtocolError): :param exc: The root-cause exception. """ def __init__(self, error_message: str, exc: Exception): - super().__init__(error_message) + super().__init__(error_message, exc) self.exc =3D exc =20 def __str__(self) -> str: @@ -76,15 +76,21 @@ class ExecuteError(QMPError): """ def __init__(self, error_response: ErrorResponse, sent: Message, received: Message): - super().__init__(error_response.error.desc) + super().__init__(error_response, sent, received) #: The sent `Message` that caused the failure self.sent: Message =3D sent #: The received `Message` that indicated failure self.received: Message =3D received #: The parsed error response self.error: ErrorResponse =3D error_response - #: The QMP error class - self.error_class: str =3D error_response.error.class_ + + @property + def error_class(self) -> str: + """The QMP error class""" + return self.error.error.class_ + + def __str__(self) -> str: + return self.error.error.desc =20 =20 class ExecInterruptedError(QMPError): @@ -110,8 +116,8 @@ class _MsgProtocolError(ProtocolError): :param error_message: Human-readable string describing the error. :param msg: The QMP `Message` that caused the error. """ - def __init__(self, error_message: str, msg: Message): - super().__init__(error_message) + def __init__(self, error_message: str, msg: Message, *args: object): + super().__init__(error_message, msg, *args) #: The received `Message` that caused the error. self.msg: Message =3D msg =20 @@ -150,7 +156,7 @@ class BadReplyError(_MsgProtocolError): :param sent: The message that was sent that prompted the error. """ def __init__(self, error_message: str, msg: Message, sent: Message): - super().__init__(error_message, msg) + super().__init__(error_message, msg, sent) #: The sent `Message` that caused the failure self.sent =3D sent =20 --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758661; cv=none; d=zohomail.com; s=zohoarc; b=I/j/ppdX5v9MX/7JpTv17PeesggPZreQnjC0JJQc90foYxami55C2FCwOmx0FYoJEWFPVs/USvH888HfsGee0JPju0Zxi967ZEtsJlQ5YqnsGjXOEahu/qU3KEsjdvtWRwMziP6ibCqQaEG3Zk6gm0wfD39V4QkhqpwttcjwgMk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758661; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YHcWkc3P5BrkTo+wY0LYOsbpNSwFaqwF1CJ9Qo0H2dw=; b=m4oD/4XzZW3nD2NV6j/sQp43ih19zUSmm17u/8/ZHzGugpCIHWG2WHkMew1rW/88sAMd5XjLesXscetSKs+kaxr1TKetePRhqH2hIiztA4eCVtEO4hq+uepmFWlqrGGG0V8aLRF40nGtZa58eT8o8JFml4Lcdkonr1lskpfQM4U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758661686484.4711342344566; Mon, 1 Sep 2025 13:31:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7b-00048E-E0; Mon, 01 Sep 2025 16:27:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7V-000467-Qp for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7U-0008Lf-Et for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:17 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-352-fZID8ix8PzSyY7jDCI8lYw-1; Mon, 01 Sep 2025 16:27:12 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C96FB19560AA; Mon, 1 Sep 2025 20:27:11 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 838BA19560AB; Mon, 1 Sep 2025 20:27:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YHcWkc3P5BrkTo+wY0LYOsbpNSwFaqwF1CJ9Qo0H2dw=; b=XELd4FnVL9CWNy9heAuX07hCpva3oUKluyW7sRZLaNWJJgE6V1oRaxS1mjvsKLFVOeVcjF Vf5FY6fHnMXtAVPLMtRukpkYqrpyCqnIsV6MUsDAwZ1YUGKeFiASClbscO5OY8gAJwK/S6 J87x2W+TldZDWxqbrtMQRU1msdxLtq8= X-MC-Unique: fZID8ix8PzSyY7jDCI8lYw-1 X-Mimecast-MFC-AGG-ID: fZID8ix8PzSyY7jDCI8lYw_1756758431 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 02/19] python: backport 'EventListener: add __repr__ method' Date: Mon, 1 Sep 2025 16:26:44 -0400 Message-ID: <20250901202702.2971212-3-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758662419116600 Content-Type: text/plain; charset="utf-8" When the object is not stateful, this repr method prints what you'd expect. In cases where there are pending events, the output is augmented to illustrate that. The object itself has no idea if it's "active" or not, so it cannot convey that information. Signed-off-by: John Snow cherry picked from commit 8a6f2e136dae395fec8aa5fd77487cfe12d9e05e --- python/qemu/qmp/events.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/python/qemu/qmp/events.py b/python/qemu/qmp/events.py index 6199776cc66..66583496192 100644 --- a/python/qemu/qmp/events.py +++ b/python/qemu/qmp/events.py @@ -497,6 +497,21 @@ def __init__( #: Optional, secondary event filter. self.event_filter: Optional[EventFilter] =3D event_filter =20 + def __repr__(self) -> str: + args: List[str] =3D [] + if self.names: + args.append(f"names=3D{self.names!r}") + if self.event_filter: + args.append(f"event_filter=3D{self.event_filter!r}") + + if self._queue.qsize(): + state =3D f"" + else: + state =3D '' + + argstr =3D ", ".join(args) + return f"{type(self).__name__}{state}({argstr})" + @property def history(self) -> Tuple[Message, ...]: """ --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758529; cv=none; d=zohomail.com; s=zohoarc; b=KpXkfjq7g0dMaYtnV7UhN1qHY6y3pNWn9z5wcmgn/Gb7G6j3rdB5SdW23kKcRpIDW/88wzKjW7zMUxmq0SAw37/cgmXKeE6an54blUaW9/piU7iLsgN7pF5Wx6HsSaZZZcxbNI8MSdXmMMmkEF+EsyPzNM5RL5Hp11sz/vWJFj0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758529; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=oitPS37GsICvVat2dGKQ6z69gkosxIhMQsYQbDyUGIU=; b=O7iEzsLQrYif/OJuMnTAkPRpJpq8+J5gM+D6pMbhRXmGGw7ZT2QVQl/QTVNReZYQCluY3z+KU5xL7tY9DVoQZnXP9o+3KOkwQyS77pYILv+VT+lTmLTUNFW9PME5ik7wv+/4DsNXJQqL8gV/7NiwxdcLBMy0NBBUI4ZMchXOEhg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758529001817.9665896777733; Mon, 1 Sep 2025 13:28:49 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7c-00048c-RQ; Mon, 01 Sep 2025 16:27:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7Z-00047T-AI for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7X-0008Ms-3r for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:21 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-201-U5HoIZ_PMoetO1L4T4lwJA-1; Mon, 01 Sep 2025 16:27:14 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F2C4B195C277; Mon, 1 Sep 2025 20:27:13 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 193A819560AB; Mon, 1 Sep 2025 20:27:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oitPS37GsICvVat2dGKQ6z69gkosxIhMQsYQbDyUGIU=; b=Yb9/MRRRpCdFBjdI13jvOCemcIiH5thd3vcR4E99dx+Be14W//oNZpL0iTPpT1FB/VLGms 8/YVOXRniyer6Is74jpW8M1qD+qdIyOEugFNE40eumD3XBynZH2qlHaFctIlXhJwLTY/SS vIVWpaqN2bERkS1o3u4vn1wAA2P6YU4= X-MC-Unique: U5HoIZ_PMoetO1L4T4lwJA-1 X-Mimecast-MFC-AGG-ID: U5HoIZ_PMoetO1L4T4lwJA_1756758434 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf , Jag Raman Subject: [PATCH 03/19] python: backport 'kick event queue on legacy event_pull()' Date: Mon, 1 Sep 2025 16:26:45 -0400 Message-ID: <20250901202702.2971212-4-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758530802116600 Content-Type: text/plain; charset="utf-8" This corrects an oversight in qmp-shell operation where new events will not accumulate in the event queue when pressing "enter" with an empty command buffer, so no new events show up. Reported-by: Jag Raman Signed-off-by: John Snow cherry picked from commit 0443582d16cf9efd52b2c41a7b5be7af42c856cd --- python/qemu/qmp/legacy.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index 22a2b5616ef..c8d0a29b56f 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -231,6 +231,9 @@ def pull_event(self, =20 :return: The first available QMP event, or None. """ + # Kick the event loop to allow events to accumulate + self._sync(asyncio.sleep(0)) + if not wait: # wait is False/0: "do not wait, do not except." if self._qmp.events.empty(): --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758671; cv=none; d=zohomail.com; s=zohoarc; b=iFdS9Ix6PxCiwz2PFsx/5lbR4uZcI1bW8bHInlTxmtTNjlmSGY/aGbjfheo+IohYZ0GpjnU0kRc0/8ugA1bKVZ1d5SilKJMEsb2UTxWeORwov0uGvsRaTmvehJijpTBW+GG3l8mpsq3qfcZKJpko50dUw2tiuPMQbctXqqlX0/w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758671; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=96fcc12r2NZGoWcAWvZ/2KcZ+/kTHQQsPcjDbDUtkY8=; b=gXk9miVsypI2Vd9xOlKPqxqKL5LnMrTauzmo3YoIK1zebgwRGC4bExIamT6Y4+JoSD8tHQKUrZXFrqYCq1DJJbwpc1kus7vBoC/5E1JXhIPq3xlNSnfD6sUn/GNBr+H5n468C2FON+MBsVG+O+KJtRS6HWB37RtvVF4++RDl/k8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758671155439.2181042687366; Mon, 1 Sep 2025 13:31:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7e-00049J-7i; Mon, 01 Sep 2025 16:27:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7b-00048U-ER for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7a-0008Ne-0o for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:23 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-VVMMJKNuMKynfl4vyFlHlQ-1; Mon, 01 Sep 2025 16:27:18 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 71FC31800446; Mon, 1 Sep 2025 20:27:17 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8C8B519560B4; Mon, 1 Sep 2025 20:27:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=96fcc12r2NZGoWcAWvZ/2KcZ+/kTHQQsPcjDbDUtkY8=; b=iryAM9CQP8V+bOzfSgjh8PBxKaamdevOLFC9KNE7Mat2sCLr/YS0ZDu/E0yfBgR+/y+U0p wiw0pb+bASEf6bAaL5fDKAKyKSpQT4UKfysQGwuqKgCCxBLCAY8wvi9MwZi6UiWYaVQCnP 5H6BHf65sAoEw6xrOM2TI2D4oEnwso8= X-MC-Unique: VVMMJKNuMKynfl4vyFlHlQ-1 X-Mimecast-MFC-AGG-ID: VVMMJKNuMKynfl4vyFlHlQ_1756758437 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 04/19] python: backport 'protocol: adjust logging name when changing client name' Date: Mon, 1 Sep 2025 16:26:46 -0400 Message-ID: <20250901202702.2971212-5-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758672619124100 Content-Type: text/plain; charset="utf-8" The client name is mutable, so the logging name should also change to reflect it when it changes. Signed-off-by: John Snow cherry picked from commit e10b73c633ce138ba30bc8beccd2ab31989eaf3d --- python/qemu/qmp/protocol.py | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py index 86e588881b7..ec4762c567b 100644 --- a/python/qemu/qmp/protocol.py +++ b/python/qemu/qmp/protocol.py @@ -217,10 +217,8 @@ class AsyncProtocol(Generic[T]): # ------------------------- =20 def __init__(self, name: Optional[str] =3D None) -> None: - #: The nickname for this connection, if any. - self.name: Optional[str] =3D name - if self.name is not None: - self.logger =3D self.logger.getChild(self.name) + self._name: Optional[str] + self.name =3D name =20 # stream I/O self._reader: Optional[StreamReader] =3D None @@ -257,6 +255,24 @@ def __repr__(self) -> str: tokens.append(f"runstate=3D{self.runstate.name}") return f"<{cls_name} {' '.join(tokens)}>" =20 + @property + def name(self) -> Optional[str]: + """ + The nickname for this connection, if any. + + This name is used for differentiating instances in debug output. + """ + return self._name + + @name.setter + def name(self, name: Optional[str]) -> None: + logger =3D logging.getLogger(__name__) + if name: + self.logger =3D logger.getChild(name) + else: + self.logger =3D logger + self._name =3D name + @property # @upper_half def runstate(self) -> Runstate: """The current `Runstate` of the connection.""" --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758557; cv=none; d=zohomail.com; s=zohoarc; b=FBrwmkk5xA3x4Ggx4bnk8WWv9NBmxixfMCHZsOycjWH0NKOcnZGQ+HmRttZLcqus91lC3sOZ3QA/JczlGkTYUj5tZbvqYqTUW0eDmvvz5w2hI+pKVh8cMhixE6tZ5ubtcpMyUwDm4f6ykDAY87q+D1HgREqkRXyLB6pSpDxZtys= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758557; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=22hO3JJMxrFCOXkNxcoFNtgJT5F0JBkIh0Au/Df0Gzw=; b=DDg7yLAOOmbcwBmxMNgAoMEbMNVjJYe2FxtwmN/4/EdgshV3uMiZ8p/2DtGixiIhg6bLK/QTl9ghyOe3wMc+1gQ3gdNISExjGlVOYHGoBDOrzRp66zOg5hMxRlCD5cEmcCMv5kD4NreXWJv2PzdxN3JkG08Tq0oQTgrF95Jgk94= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758557724582.8344490319622; Mon, 1 Sep 2025 13:29:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7h-0004BE-LZ; Mon, 01 Sep 2025 16:27:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7f-0004A5-GR for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7c-0008Qq-Oh for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:27 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-402-Bt99TanANgeoKx_E4yIv3Q-1; Mon, 01 Sep 2025 16:27:19 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0F5CA195608E; Mon, 1 Sep 2025 20:27:19 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9949519560AB; Mon, 1 Sep 2025 20:27:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758444; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=22hO3JJMxrFCOXkNxcoFNtgJT5F0JBkIh0Au/Df0Gzw=; b=ZpgUXgptK/lwojQ16tlGOKFmgN+dA/gAb9A7L1SjohqFmNvIzX+zS6QJ4oHbc3YTIAPycy L6Y9fBXoi3HsBRD2vGImQE8BfmOKE11vhDe2yigB51EQ69MPVgGpRtgE78/bwAzGUC3lzX JrQKySROvNs6yhE7iOQXt8OvCVa2a5E= X-MC-Unique: Bt99TanANgeoKx_E4yIv3Q-1 X-Mimecast-MFC-AGG-ID: Bt99TanANgeoKx_E4yIv3Q_1756758439 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 05/19] python: backport 'drop Python3.6 workarounds' Date: Mon, 1 Sep 2025 16:26:47 -0400 Message-ID: <20250901202702.2971212-6-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758559397124100 Content-Type: text/plain; charset="utf-8" Now that the minimum version is 3.7, drop some of the 3.6-specific hacks we've been carrying. A single remaining compatibility hack concerning 3.6's lack of @asynccontextmanager is addressed in the following commit. Signed-off-by: John Snow cherry picked from commit 3e8e34e594cfc6b707e6f67959166acde4b421b8 --- python/qemu/qmp/protocol.py | 48 +++++++--------- python/qemu/qmp/qmp_tui.py | 8 +-- python/qemu/qmp/util.py | 107 ++---------------------------------- python/tests/protocol.py | 8 +-- 4 files changed, 33 insertions(+), 138 deletions(-) diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py index ec4762c567b..958aeca08ac 100644 --- a/python/qemu/qmp/protocol.py +++ b/python/qemu/qmp/protocol.py @@ -15,6 +15,7 @@ =20 import asyncio from asyncio import StreamReader, StreamWriter +from contextlib import asynccontextmanager from enum import Enum from functools import wraps import logging @@ -22,6 +23,7 @@ from ssl import SSLContext from typing import ( Any, + AsyncGenerator, Awaitable, Callable, Generic, @@ -36,13 +38,10 @@ from .error import QMPError from .util import ( bottom_half, - create_task, exception_summary, flush, - is_closing, pretty_traceback, upper_half, - wait_closed, ) =20 =20 @@ -340,9 +339,8 @@ async def start_server(self, address: SocketAddrT, This exception will wrap a more concrete one. In most cases, the wrapped exception will be `OSError`. """ - await self._session_guard( - self._do_start_server(address, ssl), - 'Failed to establish connection') + async with self._session_guard('Failed to establish connection'): + await self._do_start_server(address, ssl) assert self.runstate =3D=3D Runstate.CONNECTING =20 @upper_half @@ -365,12 +363,10 @@ async def accept(self) -> None: """ if self._accepted is None: raise QMPError("Cannot call accept() before start_server().") - await self._session_guard( - self._do_accept(), - 'Failed to establish connection') - await self._session_guard( - self._establish_session(), - 'Failed to establish session') + async with self._session_guard('Failed to establish connection'): + await self._do_accept() + async with self._session_guard('Failed to establish session'): + await self._establish_session() assert self.runstate =3D=3D Runstate.RUNNING =20 @upper_half @@ -395,12 +391,10 @@ async def connect(self, address: Union[SocketAddrT, s= ocket.socket], protocol-level failure occurs while establishing a new session, the wrapped error may also be an `QMPError`. """ - await self._session_guard( - self._do_connect(address, ssl), - 'Failed to establish connection') - await self._session_guard( - self._establish_session(), - 'Failed to establish session') + async with self._session_guard('Failed to establish connection'): + await self._do_connect(address, ssl) + async with self._session_guard('Failed to establish session'): + await self._establish_session() assert self.runstate =3D=3D Runstate.RUNNING =20 @upper_half @@ -421,7 +415,8 @@ async def disconnect(self) -> None: # Section: Session machinery # -------------------------- =20 - async def _session_guard(self, coro: Awaitable[None], emsg: str) -> No= ne: + @asynccontextmanager + async def _session_guard(self, emsg: str) -> AsyncGenerator[None, None= ]: """ Async guard function used to roll back to `IDLE` on any error. =20 @@ -438,10 +433,9 @@ async def _session_guard(self, coro: Awaitable[None], = emsg: str) -> None: :raise ConnectError: When any other error is encountered in the guarded block. """ - # Note: After Python 3.6 support is removed, this should be an - # @asynccontextmanager instead of accepting a callback. try: - await coro + # Caller's code runs here. + yield except BaseException as err: self.logger.error("%s: %s", emsg, exception_summary(err)) self.logger.debug("%s:\n%s\n", emsg, pretty_traceback()) @@ -682,8 +676,8 @@ async def _establish_session(self) -> None: reader_coro =3D self._bh_loop_forever(self._bh_recv_message, 'Read= er') writer_coro =3D self._bh_loop_forever(self._bh_send_message, 'Writ= er') =20 - self._reader_task =3D create_task(reader_coro) - self._writer_task =3D create_task(writer_coro) + self._reader_task =3D asyncio.create_task(reader_coro) + self._writer_task =3D asyncio.create_task(writer_coro) =20 self._bh_tasks =3D asyncio.gather( self._reader_task, @@ -708,7 +702,7 @@ def _schedule_disconnect(self) -> None: if not self._dc_task: self._set_state(Runstate.DISCONNECTING) self.logger.debug("Scheduling disconnect.") - self._dc_task =3D create_task(self._bh_disconnect()) + self._dc_task =3D asyncio.create_task(self._bh_disconnect()) =20 @upper_half async def _wait_disconnect(self) -> None: @@ -844,13 +838,13 @@ async def _bh_close_stream(self, error_pathway: bool = =3D False) -> None: if not self._writer: return =20 - if not is_closing(self._writer): + if not self._writer.is_closing(): self.logger.debug("Closing StreamWriter.") self._writer.close() =20 self.logger.debug("Waiting for StreamWriter to close ...") try: - await wait_closed(self._writer) + await self._writer.wait_closed() except Exception: # pylint: disable=3Dbroad-except # It's hard to tell if the Stream is already closed or # not. Even if one of the tasks has failed, it may have diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index 2d9ebbd20bc..562be008d5e 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -40,7 +40,7 @@ from .message import DeserializationError, Message, UnexpectedTypeError from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient -from .util import create_task, pretty_traceback +from .util import pretty_traceback =20 =20 # The name of the signal that is used to update the history list @@ -225,7 +225,7 @@ def cb_send_to_server(self, raw_msg: str) -> None: """ try: msg =3D Message(bytes(raw_msg, encoding=3D'utf-8')) - create_task(self._send_to_server(msg)) + asyncio.create_task(self._send_to_server(msg)) except (DeserializationError, UnexpectedTypeError) as err: raw_msg =3D format_json(raw_msg) logging.info('Invalid message: %s', err.error_message) @@ -246,7 +246,7 @@ def kill_app(self) -> None: Initiates killing of app. A bridge between asynchronous and synchr= onous code. """ - create_task(self._kill_app()) + asyncio.create_task(self._kill_app()) =20 async def _kill_app(self) -> None: """ @@ -393,7 +393,7 @@ def run(self, debug: bool =3D False) -> None: handle_mouse=3DTrue, event_loop=3Devent_loop) =20 - create_task(self.manage_connection(), self.aloop) + self.aloop.create_task(self.manage_connection()) try: main_loop.run() except Exception as err: diff --git a/python/qemu/qmp/util.py b/python/qemu/qmp/util.py index ca6225e9cda..0b3e781373d 100644 --- a/python/qemu/qmp/util.py +++ b/python/qemu/qmp/util.py @@ -1,25 +1,15 @@ """ Miscellaneous Utilities =20 -This module provides asyncio utilities and compatibility wrappers for -Python 3.6 to provide some features that otherwise become available in -Python 3.7+. - -Various logging and debugging utilities are also provided, such as -`exception_summary()` and `pretty_traceback()`, used primarily for -adding information into the logging stream. +This module provides asyncio and various logging and debugging +utilities, such as `exception_summary()` and `pretty_traceback()`, used +primarily for adding information into the logging stream. """ =20 import asyncio import sys import traceback -from typing import ( - Any, - Coroutine, - Optional, - TypeVar, - cast, -) +from typing import TypeVar, cast =20 =20 T =3D TypeVar('T') @@ -79,95 +69,6 @@ def bottom_half(func: T) -> T: return func =20 =20 -# ------------------------------- -# Section: Compatibility Wrappers -# ------------------------------- - - -def create_task(coro: Coroutine[Any, Any, T], - loop: Optional[asyncio.AbstractEventLoop] =3D None - ) -> 'asyncio.Future[T]': - """ - Python 3.6-compatible `asyncio.create_task` wrapper. - - :param coro: The coroutine to execute in a task. - :param loop: Optionally, the loop to create the task in. - - :return: An `asyncio.Future` object. - """ - if sys.version_info >=3D (3, 7): - if loop is not None: - return loop.create_task(coro) - return asyncio.create_task(coro) # pylint: disable=3Dno-member - - # Python 3.6: - return asyncio.ensure_future(coro, loop=3Dloop) - - -def is_closing(writer: asyncio.StreamWriter) -> bool: - """ - Python 3.6-compatible `asyncio.StreamWriter.is_closing` wrapper. - - :param writer: The `asyncio.StreamWriter` object. - :return: `True` if the writer is closing, or closed. - """ - if sys.version_info >=3D (3, 7): - return writer.is_closing() - - # Python 3.6: - transport =3D writer.transport - assert isinstance(transport, asyncio.WriteTransport) - return transport.is_closing() - - -async def wait_closed(writer: asyncio.StreamWriter) -> None: - """ - Python 3.6-compatible `asyncio.StreamWriter.wait_closed` wrapper. - - :param writer: The `asyncio.StreamWriter` to wait on. - """ - if sys.version_info >=3D (3, 7): - await writer.wait_closed() - return - - # Python 3.6 - transport =3D writer.transport - assert isinstance(transport, asyncio.WriteTransport) - - while not transport.is_closing(): - await asyncio.sleep(0) - - # This is an ugly workaround, but it's the best I can come up with. - sock =3D transport.get_extra_info('socket') - - if sock is None: - # Our transport doesn't have a socket? ... - # Nothing we can reasonably do. - return - - while sock.fileno() !=3D -1: - await asyncio.sleep(0) - - -def asyncio_run(coro: Coroutine[Any, Any, T], *, debug: bool =3D False) ->= T: - """ - Python 3.6-compatible `asyncio.run` wrapper. - - :param coro: A coroutine to execute now. - :return: The return value from the coroutine. - """ - if sys.version_info >=3D (3, 7): - return asyncio.run(coro, debug=3Ddebug) - - # Python 3.6 - loop =3D asyncio.get_event_loop() - loop.set_debug(debug) - ret =3D loop.run_until_complete(coro) - loop.close() - - return ret - - # ---------------------------- # Section: Logging & Debugging # ---------------------------- diff --git a/python/tests/protocol.py b/python/tests/protocol.py index 56c4d441f9c..c254c77b176 100644 --- a/python/tests/protocol.py +++ b/python/tests/protocol.py @@ -8,7 +8,6 @@ =20 from qemu.qmp import ConnectError, Runstate from qemu.qmp.protocol import AsyncProtocol, StateError -from qemu.qmp.util import asyncio_run, create_task =20 =20 class NullProtocol(AsyncProtocol[None]): @@ -124,7 +123,7 @@ async def _runner(): if allow_cancellation: return raise - return create_task(_runner()) + return asyncio.create_task(_runner()) =20 =20 @contextmanager @@ -271,7 +270,7 @@ async def _watcher(): msg=3Df"Expected state '{state.name}'", ) =20 - self.runstate_watcher =3D create_task(_watcher()) + self.runstate_watcher =3D asyncio.create_task(_watcher()) # Kick the loop and force the task to block on the event. await asyncio.sleep(0) =20 @@ -589,7 +588,8 @@ async def _asyncTearDown(self): async def testSmoke(self): with TemporaryDirectory(suffix=3D'.qmp') as tmpdir: sock =3D os.path.join(tmpdir, type(self.proto).__name__ + ".so= ck") - server_task =3D create_task(self.server.start_server_and_accep= t(sock)) + server_task =3D asyncio.create_task( + self.server.start_server_and_accept(sock)) =20 # give the server a chance to start listening [...] await asyncio.sleep(0) --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758498; cv=none; d=zohomail.com; s=zohoarc; b=Om1AP7f3gzgvjkJ+DLAuQanN8B7ErhnZvXPb561U4Lyh/HHocQkMXooeUkO0ethkXzFlBR2SytyCUJ+k1YEqsjwUcqqUZPYIpSSK1hD3rWd0HSQnKOef7p80so/ufHuXZdzqfs5EY1dwoI1CHCK75mjypPSpqGCMuQ9Yx8+caBM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758498; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=vvlcoNBRtTemniW9XA73ofAIfmI7j9QtIOho6OWMmb8=; b=bB4zrJ1Twsx+Mkg2LQFyaCd+uQ/rS/kRJHQNwOyuUSu2KVpByII4VpHBBL4c4mskhzGoE8uIj2tR4gNLsShbqI0Sr4pNfTBvuno977L/AU9uV2R+Xw5wjgK/LdMD6ELnxqp6a+VqEgSssbQ+wn4WH0keQQ6K26vAOiexR5QuXGI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758498907280.66684929962753; Mon, 1 Sep 2025 13:28:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7j-0004CQ-Ca; Mon, 01 Sep 2025 16:27:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7g-0004Ad-9w for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7e-0008SM-FC for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:27 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-9J9Zpbx_P_KtHOVMdxiihw-1; Mon, 01 Sep 2025 16:27:22 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 57EE3180045C; Mon, 1 Sep 2025 20:27:21 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5234219560AB; Mon, 1 Sep 2025 20:27:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758445; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vvlcoNBRtTemniW9XA73ofAIfmI7j9QtIOho6OWMmb8=; b=dxudyT65f/nWH9jXVl4waIClkWmmHntas0if9pLGpJbA/UG+bcJ/0GPebX1aYPPNhuA2/H VW3hvM8Yd2tmKJeEma8bKzhPspu5ati/beSkKXxYDXCI/6BJBCewHo71BsPlVqT/CBAQab S6ECmK24MmTijL53FNldtjUja3pNjJM= X-MC-Unique: 9J9Zpbx_P_KtHOVMdxiihw-1 X-Mimecast-MFC-AGG-ID: 9J9Zpbx_P_KtHOVMdxiihw_1756758441 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 06/19] python: backport 'qmp-shell: add common_parser()' Date: Mon, 1 Sep 2025 16:26:48 -0400 Message-ID: <20250901202702.2971212-7-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758500844116600 Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow cherry picked from commit 20a88c2471f37d10520b2409046d59e1d0f1e905 Signed-off-by: John Snow --- python/qemu/qmp/qmp_shell.py | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py index 98e684e9e8a..02028e94b5a 100644 --- a/python/qemu/qmp/qmp_shell.py +++ b/python/qemu/qmp/qmp_shell.py @@ -514,21 +514,27 @@ def die(msg: str) -> NoReturn: sys.exit(1) =20 =20 +def common_parser() -> argparse.ArgumentParser: + """Build common parsing options used by qmp-shell and qmp-shell-wrap."= "" + parser =3D argparse.ArgumentParser() + parser.add_argument('-H', '--hmp', action=3D'store_true', + help=3D'Use HMP interface') + parser.add_argument('-v', '--verbose', action=3D'store_true', + help=3D'Verbose (echo commands sent and received)') + parser.add_argument('-p', '--pretty', action=3D'store_true', + help=3D'Pretty-print JSON') + parser.add_argument('-l', '--logfile', + help=3D'Save log of all QMP messages to PATH') + return parser + + def main() -> None: """ qmp-shell entry point: parse command line arguments and start the REPL. """ - parser =3D argparse.ArgumentParser() - parser.add_argument('-H', '--hmp', action=3D'store_true', - help=3D'Use HMP interface') + parser =3D common_parser() parser.add_argument('-N', '--skip-negotiation', action=3D'store_true', help=3D'Skip negotiate (for qemu-ga)') - parser.add_argument('-v', '--verbose', action=3D'store_true', - help=3D'Verbose (echo commands sent and received)') - parser.add_argument('-p', '--pretty', action=3D'store_true', - help=3D'Pretty-print JSON') - parser.add_argument('-l', '--logfile', - help=3D'Save log of all QMP messages to PATH') =20 default_server =3D os.environ.get('QMP_SOCKET') parser.add_argument('qmp_server', action=3D'store', @@ -564,16 +570,7 @@ def main_wrap() -> None: qmp-shell-wrap entry point: parse command line arguments and start the REPL. """ - parser =3D argparse.ArgumentParser() - parser.add_argument('-H', '--hmp', action=3D'store_true', - help=3D'Use HMP interface') - parser.add_argument('-v', '--verbose', action=3D'store_true', - help=3D'Verbose (echo commands sent and received)') - parser.add_argument('-p', '--pretty', action=3D'store_true', - help=3D'Pretty-print JSON') - parser.add_argument('-l', '--logfile', - help=3D'Save log of all QMP messages to PATH') - + parser =3D common_parser() parser.add_argument('command', nargs=3Dargparse.REMAINDER, help=3D'QEMU command line to invoke') =20 --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758672; cv=none; d=zohomail.com; s=zohoarc; b=LFBwEL4Nq/gcAi2cScKDcyA1hUdOhUDT19JRBqN4Ma0h/obYrd4+FNHEikL6rWzMP/itSXxGVeLnZJemcARA9oHHpd/n/baiXem9EuaiUHT+C6Fu1/UdVydZaPWSp/84wxwKNwYCX5hyYk0BVGwqo5suEtH9imkiREZtRStOjDg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758672; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=M6gKCmGtM4v/7K8Q4fJ0OL1dh2KHpJ2JMgXdjAgZP6A=; b=nmWGSIJJfLnvLZo1Qe491ss9Gv1claxs715v9FVMhMCAYrO2Id9A47R+g8XDeZTh3ZJRykE53UzfzXcfuLw7RTTlCFb/wz8DeRCvbiJLhnvM99VuDRaujZqzh2X6He4YK9babCJZLWtIXv2T6YSOfazSpZLGlxtjiflnLrS9Gkg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17567586721581007.0001056882254; Mon, 1 Sep 2025 13:31:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7n-0004ET-KH; Mon, 01 Sep 2025 16:27:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7i-0004Bd-Du for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7g-0008U7-Nv for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:30 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-641-wCgrJEltM5O5vYL99Plm0A-1; Mon, 01 Sep 2025 16:27:24 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8E633180047F; Mon, 1 Sep 2025 20:27:23 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5DB9919560AB; Mon, 1 Sep 2025 20:27:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M6gKCmGtM4v/7K8Q4fJ0OL1dh2KHpJ2JMgXdjAgZP6A=; b=J/DhmUeYtWoLjj0LUXyWoWzva2TMApWec1HNkEgl1RDDRWnqPLkqxgSwfB8jpVsu7m1Bgh jhqPiY3VUB9AuOPbuEccyOco7oypFd5C7/beOYOq+0JOUNa88+ZdPWHlbYXk6Xv7TQ2xVM TH+9gDIdhjLdeue5NYstJBWNqkRP1pM= X-MC-Unique: wCgrJEltM5O5vYL99Plm0A-1 X-Mimecast-MFC-AGG-ID: wCgrJEltM5O5vYL99Plm0A_1756758443 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf , Adam Dorsey , Adam Dorsey Subject: [PATCH 07/19] python: backport 'feat: allow setting read buffer limit' Date: Mon, 1 Sep 2025 16:26:49 -0400 Message-ID: <20250901202702.2971212-8-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758674915124100 Content-Type: text/plain; charset="utf-8" From: Adam Dorsey Expose the limit parameter of the underlying StreamReader and StreamWriter instances. This is helpful for the use case of transferring files in and out of a VM via the QEMU guest agent's guest-file-open, guest-file-read, guest-file-wri= te, and guest-file-close methods, as it allows pushing the buffer size up to the guest agent's limit of 48MB per transfer. Signed-off-by: Adam Dorsey cherry picked from commit 9ba6a698344eb3b570fa4864e906c54042824cd6 cherry picked from commit e4d0d3f835d82283ee0e48438d1b154e18303491 [Squashed in linter fixups. --js] Signed-off-by: John Snow --- python/qemu/qmp/protocol.py | 25 ++++++++++++++++--------- python/qemu/qmp/qmp_client.py | 18 ++++++++++++++---- 2 files changed, 30 insertions(+), 13 deletions(-) diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py index 958aeca08ac..3d5eb553aad 100644 --- a/python/qemu/qmp/protocol.py +++ b/python/qemu/qmp/protocol.py @@ -53,6 +53,9 @@ UnixAddrT =3D str SocketAddrT =3D Union[UnixAddrT, InternetAddrT] =20 +# Maximum allowable size of read buffer, default +_DEFAULT_READBUFLEN =3D 64 * 1024 + =20 class Runstate(Enum): """Protocol session runstate.""" @@ -202,22 +205,26 @@ class AsyncProtocol(Generic[T]): will log to 'qemu.qmp.protocol', but each individual connection can be given its own logger by giving it a name; messages will then log to 'qemu.qmp.protocol.${name}'. + :param readbuflen: + The maximum read buffer length of the underlying StreamReader + instance. """ # pylint: disable=3Dtoo-many-instance-attributes =20 #: Logger object for debugging messages from this connection. logger =3D logging.getLogger(__name__) =20 - # Maximum allowable size of read buffer - _limit =3D 64 * 1024 - # ------------------------- # Section: Public interface # ------------------------- =20 - def __init__(self, name: Optional[str] =3D None) -> None: + def __init__( + self, name: Optional[str] =3D None, + readbuflen: int =3D _DEFAULT_READBUFLEN + ) -> None: self._name: Optional[str] self.name =3D name + self.readbuflen =3D readbuflen =20 # stream I/O self._reader: Optional[StreamReader] =3D None @@ -574,7 +581,7 @@ async def _do_start_server(self, address: SocketAddrT, port=3Daddress[1], ssl=3Dssl, backlog=3D1, - limit=3Dself._limit, + limit=3Dself.readbuflen, ) else: coro =3D asyncio.start_unix_server( @@ -582,7 +589,7 @@ async def _do_start_server(self, address: SocketAddrT, path=3Daddress, ssl=3Dssl, backlog=3D1, - limit=3Dself._limit, + limit=3Dself.readbuflen, ) =20 # Allow runstate watchers to witness 'CONNECTING' state; some @@ -637,7 +644,7 @@ async def _do_connect(self, address: Union[SocketAddrT,= socket.socket], "fd=3D%d, family=3D%r, type=3D%r", address.fileno(), address.family, address.ty= pe) connect =3D asyncio.open_connection( - limit=3Dself._limit, + limit=3Dself.readbuflen, ssl=3Dssl, sock=3Daddress, ) @@ -647,14 +654,14 @@ async def _do_connect(self, address: Union[SocketAddr= T, socket.socket], address[0], address[1], ssl=3Dssl, - limit=3Dself._limit, + limit=3Dself.readbuflen, ) else: self.logger.debug("Connecting to file://%s ...", address) connect =3D asyncio.open_unix_connection( path=3Daddress, ssl=3Dssl, - limit=3Dself._limit, + limit=3Dself.readbuflen, ) =20 self._reader, self._writer =3D await connect diff --git a/python/qemu/qmp/qmp_client.py b/python/qemu/qmp/qmp_client.py index a87fb565ab5..d826331b6d5 100644 --- a/python/qemu/qmp/qmp_client.py +++ b/python/qemu/qmp/qmp_client.py @@ -170,6 +170,12 @@ class QMPClient(AsyncProtocol[Message], Events): =20 :param name: Optional nickname for the connection, used for logging. =20 + :param readbuflen: + The maximum buffer length for reads and writes to and from the QMP + server, in bytes. Default is 10MB. If `QMPClient` is used to + connect to a guest agent to transfer files via ``guest-file-read``/ + ``guest-file-write``, increasing this value may be required. + Basic script-style usage looks like this:: =20 qmp =3D QMPClient('my_virtual_machine_name') @@ -203,14 +209,18 @@ async def run(self, address=3D'/tmp/qemu.socket'): #: Logger object used for debugging messages. logger =3D logging.getLogger(__name__) =20 - # Read buffer limit; 10MB like libvirt default - _limit =3D 10 * 1024 * 1024 + # Read buffer default limit; 10MB like libvirt default + _readbuflen =3D 10 * 1024 * 1024 =20 # Type alias for pending execute() result items _PendingT =3D Union[Message, ExecInterruptedError] =20 - def __init__(self, name: Optional[str] =3D None) -> None: - super().__init__(name) + def __init__( + self, + name: Optional[str] =3D None, + readbuflen: int =3D _readbuflen + ) -> None: + super().__init__(name, readbuflen) Events.__init__(self) =20 #: Whether or not to await a greeting after establishing a connect= ion. --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758527; cv=none; d=zohomail.com; s=zohoarc; b=VudLpVUGq0HxLH81oeJUuYO4eNC+oZYjA9BMd2igbhAH+BnWCWspVbSv78fVleFoVFJPhr7uklzF1ZrZrOS4hwiIg/Zmi3qb73Ko5h1wVSoxhlSDYU5J63pNmP+8mGXsUb1f9gSPG0gvNkpip+8b8aE5t47SPTWTVl0sL21F77M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758527; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=4Ij7nY8K9/pQd7GIXPh8H0NEp1+Fx24DKp7yG2jE9/s=; b=Y7aeJrhXw3rOBG/lnRjzIkLaS0hRMJ2P6k2hZpTnYCi+4pPNHyPHY5HoiD12ytgAs09KaL1NLUw2PLyukcOj+qsxZvIY9fcH/gaUgpQ9aiOCLfadEptwqp0aEiBP/ByzpJBXGczmyF2QOLA2WitlpBLlceaNPdmLnejG7TPhjNQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758527070144.2969118686907; Mon, 1 Sep 2025 13:28:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7k-0004DH-PT; Mon, 01 Sep 2025 16:27:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7i-0004Bw-TP for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7h-0008UG-4M for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:30 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-357-cyrVIeDRPj226dseWauBMA-1; Mon, 01 Sep 2025 16:27:26 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9668C180044F; Mon, 1 Sep 2025 20:27:25 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D132E19560AB; Mon, 1 Sep 2025 20:27:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Ij7nY8K9/pQd7GIXPh8H0NEp1+Fx24DKp7yG2jE9/s=; b=WiemGU2NvUG6akW0AuKQYj5UiOl6yW+s3YR8dwWfHXApl1kaYAxiVkTfxG4Os/1AS/uiAx M/6PCXPDrwlCtLMJbfY5+ZbdSj4QkrnKx4VOg6kg9FwT2wikipfbD4yPF/D27P1Xt+Kwq8 4xY8xifdcA3YLeFzEFYjQ1+fVlcT9TU= X-MC-Unique: cyrVIeDRPj226dseWauBMA-1 X-Mimecast-MFC-AGG-ID: cyrVIeDRPj226dseWauBMA_1756758445 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 08/19] python: backport 'make require() preserve async-ness' Date: Mon, 1 Sep 2025 16:26:50 -0400 Message-ID: <20250901202702.2971212-9-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758528970116600 Content-Type: text/plain; charset="utf-8" This is not strictly needed functionality-wise, but doing this allows sphinx to see which decorated methods are async. Without this, sphinx misses the "async" classifier on generated docs, which ... for an async library, isn't great. It does make an already gnarly function even gnarlier, though. So, what's going on here? A synchronous function (like require() before this patch) can return a coroutine that can be awaited on, for example: def some_func(): return asyncio.task(asyncio.sleep(5)) async def some_async_func(): await some_func() However, this function is not considered to be an "async" function in the eyes of the abstract syntax tree. Specifically, some_func.__code__.co_flags will not be set with CO_COROUTINE. The interpreter uses this flag to know if it's legal to use "await" from within the body of the function. Since this function is just wrapping another function, it doesn't matter much for the decorator, but sphinx uses the stdlib inspect.iscoroutinefunction() to determine when to add the "async" prefix in generated output. This function uses the presence of CO_COROUTINE. So, in order to preserve the "async" flag for docs, the require() decorator needs to differentiate based on whether it is decorating a sync or async function and use a different wrapping mechanism accordingly. Phew. Signed-off-by: John Snow cherry picked from commit 40aa9699d619849f528032aa456dd061a4afa957 Signed-off-by: John Snow --- python/qemu/qmp/protocol.py | 53 ++++++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 21 deletions(-) diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py index 3d5eb553aad..4d8a39f014b 100644 --- a/python/qemu/qmp/protocol.py +++ b/python/qemu/qmp/protocol.py @@ -18,6 +18,7 @@ from contextlib import asynccontextmanager from enum import Enum from functools import wraps +from inspect import iscoroutinefunction import logging import socket from ssl import SSLContext @@ -130,6 +131,25 @@ def require(required_state: Runstate) -> Callable[[F],= F]: :param required_state: The `Runstate` required to invoke this method. :raise StateError: When the required `Runstate` is not met. """ + def _check(proto: 'AsyncProtocol[Any]') -> None: + name =3D type(proto).__name__ + if proto.runstate =3D=3D required_state: + return + + if proto.runstate =3D=3D Runstate.CONNECTING: + emsg =3D f"{name} is currently connecting." + elif proto.runstate =3D=3D Runstate.DISCONNECTING: + emsg =3D (f"{name} is disconnecting." + " Call disconnect() to return to IDLE state.") + elif proto.runstate =3D=3D Runstate.RUNNING: + emsg =3D f"{name} is already connected and running." + elif proto.runstate =3D=3D Runstate.IDLE: + emsg =3D f"{name} is disconnected and idle." + else: + assert False + + raise StateError(emsg, proto.runstate, required_state) + def _decorator(func: F) -> F: # _decorator is the decorator that is built by calling the # require() decorator factory; e.g.: @@ -140,29 +160,20 @@ def _decorator(func: F) -> F: @wraps(func) def _wrapper(proto: 'AsyncProtocol[Any]', *args: Any, **kwargs: Any) -> Any: - # _wrapper is the function that gets executed prior to the - # decorated method. - - name =3D type(proto).__name__ - - if proto.runstate !=3D required_state: - if proto.runstate =3D=3D Runstate.CONNECTING: - emsg =3D f"{name} is currently connecting." - elif proto.runstate =3D=3D Runstate.DISCONNECTING: - emsg =3D (f"{name} is disconnecting." - " Call disconnect() to return to IDLE state.") - elif proto.runstate =3D=3D Runstate.RUNNING: - emsg =3D f"{name} is already connected and running." - elif proto.runstate =3D=3D Runstate.IDLE: - emsg =3D f"{name} is disconnected and idle." - else: - assert False - raise StateError(emsg, proto.runstate, required_state) - # No StateError, so call the wrapped method. + _check(proto) return func(proto, *args, **kwargs) =20 - # Return the decorated method; - # Transforming Func to Decorated[Func]. + @wraps(func) + async def _async_wrapper(proto: 'AsyncProtocol[Any]', + *args: Any, **kwargs: Any) -> Any: + _check(proto) + return await func(proto, *args, **kwargs) + + # Return the decorated method; F =3D> Decorated[F] + # Use an async version when applicable, which + # preserves async signature generation in sphinx. + if iscoroutinefunction(func): + return cast(F, _async_wrapper) return cast(F, _wrapper) =20 # Return the decorator instance from the decorator factory. Phew! --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758557; cv=none; d=zohomail.com; s=zohoarc; b=hRkIhOhJSt2AAzYxcsfyUYcMSoSyb8PdaXUV16QckeNkezORaJtUICs0FG2n4GBZbgxBpMLq0F6xIPWBuH+FysrOGqJbQrIT52K493Qz/ANu0qCqe1ZJFUf2JDDEGFq4mgJHiR61gLyrLyixbe5ppmYN4f4BE7xCHYiWJWLjKII= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758557; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tagIbdrrDMrYKknepF7MVfnL4rHnC0ypp6LupSkyNl8=; b=LQlrXrqNF/ER/xgKuvTEldZIzUok1baIlqGe/db7A+ibKiuqDokgoefZ8wFJYABH9mOuR6Hn3b1Xm2P3c2A4P3fNlJd534PPHNCXKqptT91NvhSo4mbchYhdZ/ewokZQGFlL+XSVQg7ZOaIMpSylFxNtsYX3pBe8y92qWUFhees= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758557040761.2187753619803; Mon, 1 Sep 2025 13:29:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7m-0004ED-RL; Mon, 01 Sep 2025 16:27:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7k-0004D0-GM for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7j-00005K-3o for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:32 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-76-8kMckvNLPrO2lbZwa2nCjw-1; Mon, 01 Sep 2025 16:27:29 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 27DF71956086; Mon, 1 Sep 2025 20:27:28 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C06AC19560AB; Mon, 1 Sep 2025 20:27:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758450; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tagIbdrrDMrYKknepF7MVfnL4rHnC0ypp6LupSkyNl8=; b=JGzXCkEObhI1+LJJPjpOo5EbzVrJQnWdtskhE9z/ptdyQrGmUfq6uUtF1AzQjYrDugMc3N fCXLYn5Yg1ycQHR2T61LeiHZXxsHSZDHxFc98ZIbo9qbzLiM0pmktVKPed2Uxupp2qUMfN WV3QY2pIXkftNeyVyfyEfjsPE8C0EBE= X-MC-Unique: 8kMckvNLPrO2lbZwa2nCjw-1 X-Mimecast-MFC-AGG-ID: 8kMckvNLPrO2lbZwa2nCjw_1756758448 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 09/19] python: backport 'qmp-shell-wrap: handle missing binary gracefully' Date: Mon, 1 Sep 2025 16:26:51 -0400 Message-ID: <20250901202702.2971212-10-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758559154116600 Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow cherry picked from commit 9c889dcbd58817b0c917a9d2dd16161f48ac8203 Signed-off-by: John Snow --- python/qemu/qmp/qmp_shell.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py index 02028e94b5a..c923ff09e1f 100644 --- a/python/qemu/qmp/qmp_shell.py +++ b/python/qemu/qmp/qmp_shell.py @@ -607,6 +607,8 @@ def main_wrap() -> None: =20 for _ in qemu.repl(): pass + except FileNotFoundError: + sys.stderr.write(f"ERROR: QEMU executable '{cmd[0]}' not found.\n") finally: os.unlink(sockpath) =20 --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758688; cv=none; d=zohomail.com; s=zohoarc; b=bGloGMAJvbu2HrMi0rVTSsughBiG2+emcPp4dzWjq/QUrIOdx3CkLBgoJc88prBHWyQU2rCzCDFwcc0+lRtEFMKsnQP4n1xqAhCHWO14J4Dbjg6P00861eyQC821yyE6myLQpI6ZFoIstEXUp6x4IksdXTU2WNncdyMQznoVvAQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758688; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mW0ivbCeARcW77uG+hGeVTu54bG1KQkuvpiyp/K2kcE=; b=YWZjzmvoD5x30v9Qc7I0W6YlPR2zumoybbAbCpaqQY3iOYHZCV7QYMP14b8Hl3PBmOsOon5ndlIJxoNzhTLiiWnsK6ZCiqc0uEisclmOsrVrXOIqflX3O2X3zQLPgTU32SSS0PljphyDMU3QqI3rfZ5AV0TNWy9nXghqPSFKq88= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175675868867437.53043762002346; Mon, 1 Sep 2025 13:31:28 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7p-0004Fm-BA; Mon, 01 Sep 2025 16:27:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7n-0004EL-0i for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7l-00007V-4o for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:34 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-462-cH7OfPcEMbexJQsNkRTUEg-1; Mon, 01 Sep 2025 16:27:30 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 14876180047F; Mon, 1 Sep 2025 20:27:30 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5079F19560AB; Mon, 1 Sep 2025 20:27:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758452; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mW0ivbCeARcW77uG+hGeVTu54bG1KQkuvpiyp/K2kcE=; b=a/wwA1cyYtRNfQMDprrTSQjmkvUcbEI43o7q4AdqTREKkEZDlRQJCQH80cYXggSvDbUDlU 4GWpSpJ5CKlayJwNw/21UJq1rTXvyF0w+nz6UZ2BPm4VRt8z4TdrFAfob/a+xe0lq3naf3 4Yk/Uvi8/w1x+nXWTvjy/qd0PN/lNHk= X-MC-Unique: cH7OfPcEMbexJQsNkRTUEg-1 X-Mimecast-MFC-AGG-ID: cH7OfPcEMbexJQsNkRTUEg_1756758450 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 10/19] python: backport 'qmp-tui: Do not crash if optional dependencies are not met' Date: Mon, 1 Sep 2025 16:26:52 -0400 Message-ID: <20250901202702.2971212-11-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758690797116600 Content-Type: text/plain; charset="utf-8" Based on the discussion at https://github.com/pypa/pip/issues/9726 - even though the setuptools documentation implies that it is possible to guard script execution with optional dependency groups, this is not true in practice with the scripts generated by pip. Just do the simple thing and guard the import statements. Signed-off-by: John Snow cherry picked from commit df520dcacf9a75dd4c82ab1129768de4128b554c Signed-off-by: John Snow --- python/qemu/qmp/qmp_tui.py | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index 562be008d5e..53ea6c59a71 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -21,6 +21,7 @@ import logging from logging import Handler, LogRecord import signal +import sys from typing import ( List, Optional, @@ -30,10 +31,20 @@ cast, ) =20 -from pygments import lexers -from pygments import token as Token -import urwid -import urwid_readline + +try: + from pygments import lexers + from pygments import token as Token + import urwid + import urwid_readline +except ModuleNotFoundError as exc: + print( + f"Module '{exc.name}' not found.", + "You need the optional 'tui' group: pip install qemu.qmp[tui]", + sep=3D'\n', + file=3Dsys.stderr, + ) + sys.exit(1) =20 from .error import ProtocolError from .legacy import QEMUMonitorProtocol, QMPBadPortError --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758580; cv=none; d=zohomail.com; s=zohoarc; b=gGIV8pSzSR3JYvpvdd0KAMy8/b7yTsHx2IlU/sBauCc/P1lcStBgUb2iDzYlSXrMMSO7f/aUUgORMywf9LFdh2X4l/Gqpx0EmVGIJR1zZeETX3FpayKa9WtoVuSqqiWp4IphCZe4RHKhAgz3yIt1WHOKSL+TzXunkJR5wk7ZW28= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758580; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fKbaHAEvTwJqmExIXUlbRePoGgZFad/G/AMh5OouNuw=; b=GZSK6BuiHkee09ioD0xuyKgjQtTervsPQlaCa1ucbIz4rBOukQnegATnrHxS/lXsxK74jm/sTZKUNG2XngQuMyiXotgySmHmKd4m2oqlWUQklZtyzZYp/1q3AfMncH7nY8kQCcxiRo3687mDB4aVQIB3lbTS/kV2k1ZYfKHvXsg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758580748312.00071932136325; Mon, 1 Sep 2025 13:29:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7r-0004Gh-H7; Mon, 01 Sep 2025 16:27:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7q-0004GD-3M for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7o-00007z-IH for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:37 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-679-L-DDQbeHOoubFCoSSIc1aQ-1; Mon, 01 Sep 2025 16:27:33 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 108D618004D4; Mon, 1 Sep 2025 20:27:33 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 723FD19560AB; Mon, 1 Sep 2025 20:27:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKbaHAEvTwJqmExIXUlbRePoGgZFad/G/AMh5OouNuw=; b=aYpcnW8Da3YNaQeGfIwx4JtIf+A0W0xb19TpjvTt5v4TcyBVHdBrTKtm/1ihLQBVfikej6 +oHnAYYPw83+YAFuvbAzCxG9B/4mS32fz5wWOmIiI/4gMlftbpIbXEbK0swuZG6IwjVjEz qilTCJXIZbayfRtq16j2OWFjRphTd3E= X-MC-Unique: L-DDQbeHOoubFCoSSIc1aQ-1 X-Mimecast-MFC-AGG-ID: L-DDQbeHOoubFCoSSIc1aQ_1756758453 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 11/19] python: backport 'Remove deprecated get_event_loop calls' Date: Mon, 1 Sep 2025 16:26:53 -0400 Message-ID: <20250901202702.2971212-12-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758583566124100 Content-Type: text/plain; charset="utf-8" This method was deprecated in 3.12 because it ordinarily should not be used from coroutines; if there is not a currently running event loop, this automatically creates a new event loop - which is usually not what you want from code that would ever run in the bottom half. In our case, we do want this behavior in two places: (1) The synchronous shim, for convenience: this allows fully sync programs to use QEMUMonitorProtocol() without needing to set up an event loop beforehand. This is intentional to fully box in the async complexities into the legacy sync shim. (2) The qmp_tui shell; instead of relying on asyncio.run to create and run an asyncio program, we need to be able to pass the current asyncio loop to urwid setup functions. For convenience, again, we create one if one is not present to simplify the creation of the TUI appliance. The remaining user of get_event_loop() was in fact one of the erroneous users that should not have been using this function: if there's no running event loop inside of a coroutine, you're in big trouble :) Signed-off-by: John Snow cherry picked from commit aa1ff9907603a3033296027e1bd021133df86ef1 Signed-off-by: John Snow --- python/qemu/qmp/legacy.py | 9 ++++++++- python/qemu/qmp/qmp_tui.py | 7 ++++++- python/tests/protocol.py | 2 +- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index c8d0a29b56f..735d42971e9 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -86,7 +86,14 @@ def __init__(self, "server argument should be False when passing a socket") =20 self._qmp =3D QMPClient(nickname) - self._aloop =3D asyncio.get_event_loop() + + try: + self._aloop =3D asyncio.get_running_loop() + except RuntimeError: + # No running loop; since this is a sync shim likely to be + # used in fully sync programs, create one if neccessary. + self._aloop =3D asyncio.get_event_loop_policy().get_event_loop= () + self._address =3D address self._timeout: Optional[float] =3D None =20 diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index 53ea6c59a71..12bdc17c99e 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -388,7 +388,12 @@ def run(self, debug: bool =3D False) -> None: screen =3D urwid.raw_display.Screen() screen.set_terminal_properties(256) =20 - self.aloop =3D asyncio.get_event_loop() + try: + self.aloop =3D asyncio.get_running_loop() + except RuntimeError: + # No running asyncio event loop. Create one if necessary. + self.aloop =3D asyncio.get_event_loop_policy().get_event_loop() + self.aloop.set_debug(debug) =20 # Gracefully handle SIGTERM and SIGINT signals diff --git a/python/tests/protocol.py b/python/tests/protocol.py index c254c77b176..e565802516d 100644 --- a/python/tests/protocol.py +++ b/python/tests/protocol.py @@ -227,7 +227,7 @@ def async_test(async_test_method): Decorator; adds SetUp and TearDown to async tests. """ async def _wrapper(self, *args, **kwargs): - loop =3D asyncio.get_event_loop() + loop =3D asyncio.get_running_loop() loop.set_debug(True) =20 await self._asyncSetUp() --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758536; cv=none; d=zohomail.com; s=zohoarc; b=jZMGGZwgS6LuFi0rimIkS+ic77rvQcXqBZKFvleYYRFduwcw28f2lt11WqBZS19BTBnDG6NWf8JnDiV/DMVCBx7F8XE/k4y4mNvZ1b2MaZz0oluMlu4MHQE07owB5c1zSBopZCp3Q+fae4/np04+8/9CraQIHoEv04EJpr4VWdY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758536; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Se0q5+uT6KWiejFo7/IaGRao5fvt636my4wPKmetJz0=; b=NLVuCGTQ2mvEwKAA5686fAk64tbaFhpWroZiNUlKQkGC8AqSV9mho/a5MHtEtk5ckVFKgAcNvKahQ4mGEzrY2o0q9BVeKZ9c73IYYS/SeARLRDFKQWUoYjPemcp/aMPiisEd6jN6qPFDKJR0mldBanrrSIniYI/sMGwPz0N5qcE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758536031338.7713168939813; Mon, 1 Sep 2025 13:28:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7v-0004Hb-BL; Mon, 01 Sep 2025 16:27:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7s-0004H8-Ld for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7r-00008U-25 for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:40 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-180-Z9gLCwudODqDvUw21qrhJA-1; Mon, 01 Sep 2025 16:27:37 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2ABF0180035F; Mon, 1 Sep 2025 20:27:36 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6C76319560B4; Mon, 1 Sep 2025 20:27:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Se0q5+uT6KWiejFo7/IaGRao5fvt636my4wPKmetJz0=; b=KvGxIfCeVcEN1dVbSnAWOpYVXNh5Wx4o6Pcmj0PDe85ZM4HmpSSJo+jfjc4qsDLZaO+73F gqsQlxjxjUSkR3L7Q5O3zFAhqWZkmKI93JRCs6jeiZf3zMhkpozy+Oy7yGHwv6S5TNTJ5J ew/KHf1ku+obBP6awQsaf1zm11Weoj0= X-MC-Unique: Z9gLCwudODqDvUw21qrhJA-1 X-Mimecast-MFC-AGG-ID: Z9gLCwudODqDvUw21qrhJA_1756758456 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf , "Richard W.M. Jones" Subject: [PATCH 12/19] python: backport '*really* remove get_event_loop' Date: Mon, 1 Sep 2025 16:26:54 -0400 Message-ID: <20250901202702.2971212-13-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758539423124100 A prior commit, aa1ff990, switched away from using get_event_loop *by default*, but this is not good enough to avoid deprecation warnings as `asyncio.get_event_loop_policy().get_event_loop()` is *also* deprecated. Replace this mechanism with explicit calls to asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to match. Reported-by: Richard W.M. Jones Reported-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow cherry picked from commit 21ce2ee4f2df87efe84a27b9c5112487f4670622 Signed-off-by: John Snow --- python/qemu/qmp/legacy.py | 47 +++++++++++++++++++++++++++----------- python/qemu/qmp/qmp_tui.py | 10 ++++++-- 2 files changed, 42 insertions(+), 15 deletions(-) diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index 735d42971e9..775b1fdd3b3 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -86,13 +86,15 @@ def __init__(self, "server argument should be False when passing a socket") =20 self._qmp =3D QMPClient(nickname) + self._created_loop =3D False =20 try: self._aloop =3D asyncio.get_running_loop() except RuntimeError: - # No running loop; since this is a sync shim likely to be - # used in fully sync programs, create one if neccessary. - self._aloop =3D asyncio.get_event_loop_policy().get_event_loop= () + # No running loop; since this is a sync shim likely to be used + # in sync programs without any event loop at all, create one. + self._aloop =3D asyncio.new_event_loop() + self._created_loop =3D True =20 self._address =3D address self._timeout: Optional[float] =3D None @@ -313,17 +315,36 @@ def send_fd_scm(self, fd: int) -> None: self._qmp.send_fd_scm(fd) =20 def __del__(self) -> None: - if self._qmp.runstate =3D=3D Runstate.IDLE: - return + if self._qmp.runstate !=3D Runstate.IDLE: + self._qmp.logger.warning( + "QEMUMonitorProtocol object garbage collected without a pr= ior " + "call to close()" + ) =20 if not self._aloop.is_running(): - self.close() - else: - # Garbage collection ran while the event loop was running. - # Nothing we can do about it now, but if we don't raise our - # own error, the user will be treated to a lot of traceback - # they might not understand. + if self._qmp.runstate !=3D Runstate.IDLE: + # If the user neglected to close the QMP session and we + # are not currently running in an asyncio context, we + # have the opportunity to close the QMP session. If we + # do not do this, the error messages presented over + # dangling async resources may not make any sense to the + # user. + self.close() + + # If we created our own loop (and we are not running inside + # of it), we must close it to avoid warnings and error + # messages upon program exit. + if self._created_loop: + self._aloop.close() + + if self._qmp.runstate !=3D Runstate.IDLE: + # If QMP is still not quiesced, it means that the garbage + # collector ran from a context within the event loop and we + # are simply too late to take any corrective action. Raise + # our own error to give meaningful feedback to the user in + # order to prevent pages of asyncio stacktrace jargon. raise QMPError( - "QEMUMonitorProtocol.close()" - " was not called before object was garbage collected" + "QEMUMonitorProtocol.close() was not called before object = was " + "garbage collected, and could not be closed due to GC runn= ing " + "in the event loop" ) diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index 12bdc17c99e..d5526338f22 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -161,6 +161,7 @@ def __init__(self, address: Union[str, Tuple[str, int]]= , num_retries: int, self.retry_delay =3D retry_delay if retry_delay else 2 self.retry: bool =3D False self.exiting: bool =3D False + self._created_loop =3D False super().__init__() =20 def add_to_history(self, msg: str, level: Optional[str] =3D None) -> N= one: @@ -391,8 +392,9 @@ def run(self, debug: bool =3D False) -> None: try: self.aloop =3D asyncio.get_running_loop() except RuntimeError: - # No running asyncio event loop. Create one if necessary. - self.aloop =3D asyncio.get_event_loop_policy().get_event_loop() + # No running asyncio event loop. Create one. + self.aloop =3D asyncio.new_event_loop() + self._created_loop =3D True =20 self.aloop.set_debug(debug) =20 @@ -416,6 +418,10 @@ def run(self, debug: bool =3D False) -> None: logging.error('%s\n%s\n', str(err), pretty_traceback()) raise err =20 + def __del__(self) -> None: + if self._created_loop and self.aloop: + self.aloop.close() + =20 class StatusBar(urwid.Text): """ --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758566; cv=none; d=zohomail.com; s=zohoarc; b=JPGHt77oupj0cdviDzU+1uSinPb2HGcDHl6aNTAqN9kgiDJZKXxx7O8Joq1kKz3Fwe2zt0eYNOp9LBE3zQSlDXzIsRCn7/sL+paFpf89imL0ZVI2+RwWGi9bb5j4KAsPNKEOyndjeycdWM8CnCXFdUD2IfICe6o1kPdlB/7XkiE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758566; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YZgCm4hCp31KslUyaQcGicvCOl8n+vAe/zDCm9bBpQg=; b=esE5X/v2WIyK8h12+PhyCjpsMOj+UURMaAqCspg/VwGt2Nf26ZYxLSjuckAeQ5CSBqIX1TM9MucvOfptJ5cOvQOtveuFuKrudl94Wa0fjR81FE2RZTDt2t5p3mnyhhp2CCYc+v1wn1C6qygPGXWxc53sgSEZoV/kjyYotNS/Aa0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175675856601767.21004582603553; Mon, 1 Sep 2025 13:29:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB7z-0004JK-1B; Mon, 01 Sep 2025 16:27:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7w-0004IC-Ib for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7u-00009Q-HJ for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:44 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-37-7pxelDCsMX-SzmtFAtoSaQ-1; Mon, 01 Sep 2025 16:27:39 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B81881956089; Mon, 1 Sep 2025 20:27:38 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AAA3A19560AB; Mon, 1 Sep 2025 20:27:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YZgCm4hCp31KslUyaQcGicvCOl8n+vAe/zDCm9bBpQg=; b=EJkv01AXFsoiEfyKlowAdEKS7GZqsEf4Bj7+M9046VV0GtrunJBuznUazdtg+vCwFY3ZL0 KbH0FolV3XCDI5v0/cQPFs3MZ0zNXnVzNBs0/iFyQLpj9wYqYJY85o2gloNEtondh971GE rPzGMYIRlvjWfW7tv+IB2tLDEft26Q0= X-MC-Unique: 7pxelDCsMX-SzmtFAtoSaQ-1 X-Mimecast-MFC-AGG-ID: 7pxelDCsMX-SzmtFAtoSaQ_1756758458 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 13/19] python: backport 'python: avoid creating additional event loops per thread' Date: Mon, 1 Sep 2025 16:26:55 -0400 Message-ID: <20250901202702.2971212-14-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758567468124100 Content-Type: text/plain; charset="utf-8" "Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated behavior altogether by calling new_event_loop() directly if there was no loop currently running, but this has the unfortunate side effect of potentially creating multiple event loops per thread if tests instantiate multiple QMP connections in a single thread. This behavior is apparently not well-defined and causes problems in some, but not all, combinations of Python interpreter version and platform environment. Partially revert to Daniel Berrange's original patch, which calls get_event_loop and simply suppresses the deprecation warning in Python<=3D3.13. This time, however, additionally register new loops created with new_event_loop() so that future calls to get_event_loop() will return the loop already created. Signed-off-by: John Snow cherry picked from commit c08fb82b38212956ccffc03fc6d015c3979f42fe Signed-off-by: John Snow --- python/qemu/qmp/legacy.py | 21 +++++---------------- python/qemu/qmp/qmp_tui.py | 16 ++-------------- python/qemu/qmp/util.py | 27 +++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 30 deletions(-) diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index 775b1fdd3b3..e46695ae2c8 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -38,6 +38,7 @@ from .error import QMPError from .protocol import Runstate, SocketAddrT from .qmp_client import QMPClient +from .util import get_or_create_event_loop =20 =20 #: QMPMessage is an entire QMP message of any kind. @@ -86,19 +87,13 @@ def __init__(self, "server argument should be False when passing a socket") =20 self._qmp =3D QMPClient(nickname) - self._created_loop =3D False - - try: - self._aloop =3D asyncio.get_running_loop() - except RuntimeError: - # No running loop; since this is a sync shim likely to be used - # in sync programs without any event loop at all, create one. - self._aloop =3D asyncio.new_event_loop() - self._created_loop =3D True - self._address =3D address self._timeout: Optional[float] =3D None =20 + # This is a sync shim intended for use in fully synchronous + # programs. Create and set an event loop if necessary. + self._aloop =3D get_or_create_event_loop() + if server: assert not isinstance(self._address, socket.socket) self._sync(self._qmp.start_server(self._address)) @@ -331,12 +326,6 @@ def __del__(self) -> None: # user. self.close() =20 - # If we created our own loop (and we are not running inside - # of it), we must close it to avoid warnings and error - # messages upon program exit. - if self._created_loop: - self._aloop.close() - if self._qmp.runstate !=3D Runstate.IDLE: # If QMP is still not quiesced, it means that the garbage # collector ran from a context within the event loop and we diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py index d5526338f22..d946c205131 100644 --- a/python/qemu/qmp/qmp_tui.py +++ b/python/qemu/qmp/qmp_tui.py @@ -51,7 +51,7 @@ from .message import DeserializationError, Message, UnexpectedTypeError from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient -from .util import pretty_traceback +from .util import get_or_create_event_loop, pretty_traceback =20 =20 # The name of the signal that is used to update the history list @@ -161,7 +161,6 @@ def __init__(self, address: Union[str, Tuple[str, int]]= , num_retries: int, self.retry_delay =3D retry_delay if retry_delay else 2 self.retry: bool =3D False self.exiting: bool =3D False - self._created_loop =3D False super().__init__() =20 def add_to_history(self, msg: str, level: Optional[str] =3D None) -> N= one: @@ -388,14 +387,7 @@ def run(self, debug: bool =3D False) -> None: """ screen =3D urwid.raw_display.Screen() screen.set_terminal_properties(256) - - try: - self.aloop =3D asyncio.get_running_loop() - except RuntimeError: - # No running asyncio event loop. Create one. - self.aloop =3D asyncio.new_event_loop() - self._created_loop =3D True - + self.aloop =3D get_or_create_event_loop() self.aloop.set_debug(debug) =20 # Gracefully handle SIGTERM and SIGINT signals @@ -418,10 +410,6 @@ def run(self, debug: bool =3D False) -> None: logging.error('%s\n%s\n', str(err), pretty_traceback()) raise err =20 - def __del__(self) -> None: - if self._created_loop and self.aloop: - self.aloop.close() - =20 class StatusBar(urwid.Text): """ diff --git a/python/qemu/qmp/util.py b/python/qemu/qmp/util.py index 0b3e781373d..47ec39a8b5e 100644 --- a/python/qemu/qmp/util.py +++ b/python/qemu/qmp/util.py @@ -10,6 +10,7 @@ import sys import traceback from typing import TypeVar, cast +import warnings =20 =20 T =3D TypeVar('T') @@ -20,6 +21,32 @@ # -------------------------- =20 =20 +def get_or_create_event_loop() -> asyncio.AbstractEventLoop: + """ + Return this thread's current event loop, or create a new one. + + This function behaves similarly to asyncio.get_event_loop() in + Python<=3D3.13, where if there is no event loop currently associated + with the current context, it will create and register one. It should + generally not be used in any asyncio-native applications. + """ + try: + with warnings.catch_warnings(): + # Python <=3D 3.13 will trigger deprecation warnings if no + # event loop is set, but will create and set a new loop. + warnings.simplefilter("ignore") + loop =3D asyncio.get_event_loop() + except RuntimeError: + # Python 3.14+: No event loop set for this thread, + # create and set one. + loop =3D asyncio.new_event_loop() + # Set this loop as the current thread's loop, to be returned + # by calls to get_event_loop() in the future. + asyncio.set_event_loop(loop) + + return loop + + async def flush(writer: asyncio.StreamWriter) -> None: """ Utility function to ensure a StreamWriter is *fully* drained. --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758540; cv=none; d=zohomail.com; s=zohoarc; b=YXQWOCXXksOreNheuZ7Pzn0qBq6LsYi17cPMVClInE5l67vQF9YtLcvQKe2MRc9Y/41gIrRBlk438+FWabmlrzOinW2MsUWzRdCQuiX3GDdjGWRBAkMF/rxAmWJnwju+tXIBomfJQvoLCPgE5+xOyMpZKFpXvXtkb+8s+zKh7P0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758540; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TxO4lBmMxGa04h84r5x3imc7ofF7t8ngOEeqErnHQGE=; b=Seewh6bM3t0sM7hG965WR2HwF2Rmnekav191BTXXZ/GbvmkL9bDYDCxGORVBQcKDhDJUgdFrMkPt7xaM+sjJ+YxKJRYpDrX2UO5gIxjvzxcTkxttxezEJGfwJb2/dRt7DhVrfViI29LAfmNAeowK3wnxdaCu+WTtneBiGkLjYWc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758540506198.67173073935157; Mon, 1 Sep 2025 13:29:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB91-0004vT-Ku; Mon, 01 Sep 2025 16:28:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB81-0004Lt-A6 for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7y-0000Bh-3j for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:49 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-685-z5fE5Fv5ONysvrRkuAOVaQ-1; Mon, 01 Sep 2025 16:27:42 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 74AE21956086; Mon, 1 Sep 2025 20:27:41 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1498B19560B4; Mon, 1 Sep 2025 20:27:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758465; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TxO4lBmMxGa04h84r5x3imc7ofF7t8ngOEeqErnHQGE=; b=QrpidM/MOmkjkyBomCRebutghmKgVII/MichloIWOa23WpQD1JikOthPtFT0vvd0bmjOix t1ypr8e52mLD9wl2Ja4YdtlTMIOj8L6PMgoXxErdWL7rFEBnrji3i6Tmh2BcwyC5Lvsaj5 7QcvGXbEpoXJDOsEBM6fBQT4ONmTgWU= X-MC-Unique: z5fE5Fv5ONysvrRkuAOVaQ-1 X-Mimecast-MFC-AGG-ID: z5fE5Fv5ONysvrRkuAOVaQ_1756758461 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 14/19] python: synchronize qemu.qmp documentation Date: Mon, 1 Sep 2025 16:26:56 -0400 Message-ID: <20250901202702.2971212-15-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758543228116600 This patch collects comments and documentation changes from many commits in the python-qemu-qmp repository; bringing the qemu.git copy in bit-identical alignment with the standalone library *except* for several copyright messages that reference the "LICENSE" file which is, for QEMU, named "COPYING" instead and are therefore left unchanged. Signed-off-by: John Snow --- python/qemu/qmp/__init__.py | 3 +- python/qemu/qmp/events.py | 35 +++++++--- python/qemu/qmp/legacy.py | 4 +- python/qemu/qmp/message.py | 10 ++- python/qemu/qmp/models.py | 8 +-- python/qemu/qmp/protocol.py | 37 ++++++---- python/qemu/qmp/qmp_client.py | 117 +++++++++++++++++++++++-------- python/qemu/qmp/qmp_shell.py | 128 ++++++++++++++++++++++++++-------- python/qemu/qmp/util.py | 9 ++- 9 files changed, 264 insertions(+), 87 deletions(-) diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py index 69190d057a5..058139dc3ca 100644 --- a/python/qemu/qmp/__init__.py +++ b/python/qemu/qmp/__init__.py @@ -39,7 +39,8 @@ logging.getLogger('qemu.qmp').addHandler(logging.NullHandler()) =20 =20 -# The order of these fields impact the Sphinx documentation order. +# IMPORTANT: When modifying this list, update the Sphinx overview docs. +# Anything visible in the qemu.qmp namespace should be on the overview pag= e. __all__ =3D ( # Classes, most to least important 'QMPClient', diff --git a/python/qemu/qmp/events.py b/python/qemu/qmp/events.py index 66583496192..cfb5f0ac621 100644 --- a/python/qemu/qmp/events.py +++ b/python/qemu/qmp/events.py @@ -12,7 +12,14 @@ ---------------------- =20 In all of the following examples, we assume that we have a `QMPClient` -instantiated named ``qmp`` that is already connected. +instantiated named ``qmp`` that is already connected. For example: + +.. code:: python + + from qemu.qmp import QMPClient + + qmp =3D QMPClient('example-vm') + await qmp.connect('127.0.0.1', 1234) =20 =20 `listener()` context blocks with one name @@ -87,7 +94,9 @@ event =3D listener.get() print(f"Event arrived: {event['event']}") =20 -This event stream will never end, so these blocks will never terminate. +This event stream will never end, so these blocks will never +terminate. Even if the QMP connection errors out prematurely, this +listener will go silent without raising an error. =20 =20 Using asyncio.Task to concurrently retrieve events @@ -227,16 +236,20 @@ async def print_events(listener): .. code:: python =20 await qmp.execute('stop') - qmp.events.clear() + discarded =3D qmp.events.clear() await qmp.execute('cont') event =3D await qmp.events.get() assert event['event'] =3D=3D 'RESUME' + assert discarded[0]['event'] =3D=3D 'STOP' =20 `EventListener` objects are FIFO queues. If events are not consumed, they will remain in the queue until they are witnessed or discarded via `clear()`. FIFO queues will be drained automatically upon leaving a context block, or when calling `remove_listener()`. =20 +Any events removed from the queue in this fashion will be returned by +the clear call. + =20 Accessing listener history ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -350,6 +363,12 @@ def filter(event: Message) -> bool: break =20 =20 +Note that in the above example, we explicitly wait on jobA to conclude +first, and then wait for jobB to do the same. All we have guaranteed is +that the code that waits for jobA will not accidentally consume the +event intended for the jobB waiter. + + Extending the `EventListener` class ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 @@ -407,13 +426,13 @@ def accept(self, event) -> bool: These interfaces are not ones I am sure I will keep or otherwise modify heavily. =20 -qmp.listener()=E2=80=99s type signature +qmp.listen()=E2=80=99s type signature ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -`listener()` does not return anything, because it was assumed the caller +`listen()` does not return anything, because it was assumed the caller already had a handle to the listener. However, for -``qmp.listener(EventListener())`` forms, the caller will not have saved -a handle to the listener. +``qmp.listen(EventListener())`` forms, the caller will not have saved a +handle to the listener. =20 Because this function can accept *many* listeners, I found it hard to accurately type in a way where it could be used in both =E2=80=9Cone=E2=80= =9D or =E2=80=9Cmany=E2=80=9D @@ -633,7 +652,7 @@ class Events: def __init__(self) -> None: self._listeners: List[EventListener] =3D [] =20 - #: Default, all-events `EventListener`. + #: Default, all-events `EventListener`. See `qmp.events` for more = info. self.events: EventListener =3D EventListener() self.register_listener(self.events) =20 diff --git a/python/qemu/qmp/legacy.py b/python/qemu/qmp/legacy.py index e46695ae2c8..060ed0eb9d4 100644 --- a/python/qemu/qmp/legacy.py +++ b/python/qemu/qmp/legacy.py @@ -293,8 +293,8 @@ def settimeout(self, timeout: Optional[float]) -> None: """ Set the timeout for QMP RPC execution. =20 - This timeout affects the `cmd`, `cmd_obj`, and `command` methods. - The `accept`, `pull_event` and `get_event` methods have their + This timeout affects the `cmd`, `cmd_obj`, and `cmd_raw` methods. + The `accept`, `pull_event` and `get_events` methods have their own configurable timeouts. =20 :param timeout: diff --git a/python/qemu/qmp/message.py b/python/qemu/qmp/message.py index c2e9dd0dd54..dabb8ec360e 100644 --- a/python/qemu/qmp/message.py +++ b/python/qemu/qmp/message.py @@ -28,7 +28,8 @@ class Message(MutableMapping[str, object]): be instantiated from either another mapping (like a `dict`), or from raw `bytes` that still need to be deserialized. =20 - Once instantiated, it may be treated like any other MutableMapping:: + Once instantiated, it may be treated like any other + :py:obj:`~collections.abc.MutableMapping`:: =20 >>> msg =3D Message(b'{"hello": "world"}') >>> assert msg['hello'] =3D=3D 'world' @@ -50,12 +51,19 @@ class Message(MutableMapping[str, object]): >>> dict(msg) {'hello': 'world'} =20 + Or pretty-printed:: + + >>> print(str(msg)) + { + "hello": "world" + } =20 :param value: Initial value, if any. :param eager: When `True`, attempt to serialize or deserialize the initial value immediately, so that conversion exceptions are raised during the call to ``__init__()``. + """ # pylint: disable=3Dtoo-many-ancestors =20 diff --git a/python/qemu/qmp/models.py b/python/qemu/qmp/models.py index da52848d5a7..7e0d0baf038 100644 --- a/python/qemu/qmp/models.py +++ b/python/qemu/qmp/models.py @@ -54,7 +54,7 @@ def __repr__(self) -> str: =20 class Greeting(Model): """ - Defined in qmp-spec.rst, section "Server Greeting". + Defined in `interop/qmp-spec`, "Server Greeting" section. =20 :param raw: The raw Greeting object. :raise KeyError: If any required fields are absent. @@ -82,7 +82,7 @@ def _asdict(self) -> Dict[str, object]: =20 class QMPGreeting(Model): """ - Defined in qmp-spec.rst, section "Server Greeting". + Defined in `interop/qmp-spec`, "Server Greeting" section. =20 :param raw: The raw QMPGreeting object. :raise KeyError: If any required fields are absent. @@ -104,7 +104,7 @@ def __init__(self, raw: Mapping[str, Any]): =20 class ErrorResponse(Model): """ - Defined in qmp-spec.rst, section "Error". + Defined in `interop/qmp-spec`, "Error" section. =20 :param raw: The raw ErrorResponse object. :raise KeyError: If any required fields are absent. @@ -126,7 +126,7 @@ def __init__(self, raw: Mapping[str, Any]): =20 class ErrorInfo(Model): """ - Defined in qmp-spec.rst, section "Error". + Defined in `interop/qmp-spec`, "Error" section. =20 :param raw: The raw ErrorInfo object. :raise KeyError: If any required fields are absent. diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py index 4d8a39f014b..219d092a792 100644 --- a/python/qemu/qmp/protocol.py +++ b/python/qemu/qmp/protocol.py @@ -79,6 +79,12 @@ class ConnectError(QMPError): This Exception always wraps a "root cause" exception that can be interrogated for additional information. =20 + For example, when connecting to a non-existent socket:: + + await qmp.connect('not_found.sock') + # ConnectError: Failed to establish connection: + # [Errno 2] No such file or directory + :param error_message: Human-readable string describing the error. :param exc: The root-cause exception. """ @@ -102,8 +108,8 @@ class StateError(QMPError): An API command (connect, execute, etc) was issued at an inappropriate = time. =20 This error is raised when a command like - :py:meth:`~AsyncProtocol.connect()` is issued at an inappropriate - time. + :py:meth:`~AsyncProtocol.connect()` is called when the client is + already connected. =20 :param error_message: Human-readable string describing the state viola= tion. :param state: The actual `Runstate` seen at the time of the violation. @@ -298,7 +304,7 @@ def runstate(self) -> Runstate: @upper_half async def runstate_changed(self) -> Runstate: """ - Wait for the `runstate` to change, then return that runstate. + Wait for the `runstate` to change, then return that `Runstate`. """ await self._runstate_event.wait() return self.runstate @@ -312,9 +318,9 @@ async def start_server_and_accept( """ Accept a connection and begin processing message queues. =20 - If this call fails, `runstate` is guaranteed to be set back to `ID= LE`. - This method is precisely equivalent to calling `start_server()` - followed by `accept()`. + If this call fails, `runstate` is guaranteed to be set back to + `IDLE`. This method is precisely equivalent to calling + `start_server()` followed by :py:meth:`~AsyncProtocol.accept()`. =20 :param address: Address to listen on; UNIX socket path or TCP address/port. @@ -327,7 +333,8 @@ async def start_server_and_accept( This exception will wrap a more concrete one. In most cases, the wrapped exception will be `OSError` or `EOFError`. If a protocol-level failure occurs while establishing a new - session, the wrapped error may also be an `QMPError`. + session, the wrapped error may also be a `QMPError`. + """ await self.start_server(address, ssl) await self.accept() @@ -343,8 +350,8 @@ async def start_server(self, address: SocketAddrT, This method starts listening for an incoming connection, but does not block waiting for a peer. This call will return immediately after binding and listening on a socket. A later - call to `accept()` must be made in order to finalize the - incoming connection. + call to :py:meth:`~AsyncProtocol.accept()` must be made in order + to finalize the incoming connection. =20 :param address: Address to listen on; UNIX socket path or TCP address/port. @@ -367,10 +374,12 @@ async def accept(self) -> None: """ Accept an incoming connection and begin processing message queues. =20 - If this call fails, `runstate` is guaranteed to be set back to `ID= LE`. + Used after a previous call to `start_server()` to accept an + incoming connection. If this call fails, `runstate` is + guaranteed to be set back to `IDLE`. =20 :raise StateError: When the `Runstate` is not `CONNECTING`. - :raise QMPError: When `start_server()` was not called yet. + :raise QMPError: When `start_server()` was not called first. :raise ConnectError: When a connection or session cannot be established. =20 @@ -423,7 +432,11 @@ async def disconnect(self) -> None: If there was an exception that caused the reader/writers to terminate prematurely, it will be raised here. =20 - :raise Exception: When the reader or writer terminate unexpectedly. + :raise Exception: + When the reader or writer terminate unexpectedly. You can + expect to see `EOFError` if the server hangs up, or + `OSError` for connection-related issues. If there was a QMP + protocol-level problem, `ProtocolError` will be seen. """ self.logger.debug("disconnect() called.") self._schedule_disconnect() diff --git a/python/qemu/qmp/qmp_client.py b/python/qemu/qmp/qmp_client.py index d826331b6d5..8beccfe29d3 100644 --- a/python/qemu/qmp/qmp_client.py +++ b/python/qemu/qmp/qmp_client.py @@ -70,6 +70,17 @@ class ExecuteError(QMPError): """ Exception raised by `QMPClient.execute()` on RPC failure. =20 + This exception is raised when the server received, interpreted, and + replied to a command successfully; but the command itself returned a + failure status. + + For example:: + + await qmp.execute('block-dirty-bitmap-add', + {'node': 'foo', 'name': 'my_bitmap'}) + # qemu.qmp.qmp_client.ExecuteError: + # Cannot find device=3D'foo' nor node-name=3D'foo' + :param error_response: The RPC error response object. :param sent: The sent RPC message that caused the failure. :param received: The raw RPC error reply received. @@ -99,9 +110,22 @@ class ExecInterruptedError(QMPError): =20 This error is raised when an `execute()` statement could not be completed. This can occur because the connection itself was - terminated before a reply was received. + terminated before a reply was received. The true cause of the + interruption will be available via `disconnect()`. =20 - The true cause of the interruption will be available via `disconnect()= `. + The QMP protocol does not make it possible to know if a command + succeeded or failed after such an event; the client will need to + query the server to determine the state of the server on a + case-by-case basis. + + For example, ECONNRESET might look like this:: + + try: + await qmp.execute('query-block') + # ExecInterruptedError: Disconnected + except ExecInterruptedError: + await qmp.disconnect() + # ConnectionResetError: [Errno 104] Connection reset by peer """ =20 =20 @@ -162,13 +186,14 @@ def __init__(self, error_message: str, msg: Message, = sent: Message): =20 =20 class QMPClient(AsyncProtocol[Message], Events): - """ - Implements a QMP client connection. + """Implements a QMP client connection. =20 - QMP can be used to establish a connection as either the transport - client or server, though this class always acts as the QMP client. + `QMPClient` can be used to either connect or listen to a QMP server, + but always acts as the QMP client. =20 - :param name: Optional nickname for the connection, used for logging. + :param name: + Optional nickname for the connection, used to differentiate + instances when logging. =20 :param readbuflen: The maximum buffer length for reads and writes to and from the QMP @@ -178,14 +203,21 @@ class QMPClient(AsyncProtocol[Message], Events): =20 Basic script-style usage looks like this:: =20 - qmp =3D QMPClient('my_virtual_machine_name') - await qmp.connect(('127.0.0.1', 1234)) - ... - res =3D await qmp.execute('block-query') - ... - await qmp.disconnect() + import asyncio + from qemu.qmp import QMPClient =20 - Basic async client-style usage looks like this:: + async def main(): + qmp =3D QMPClient('my_virtual_machine_name') + await qmp.connect(('127.0.0.1', 1234)) + ... + res =3D await qmp.execute('query-block') + ... + await qmp.disconnect() + + asyncio.run(main()) + + A more advanced example that starts to take advantage of asyncio + might look like this:: =20 class Client: def __init__(self, name: str): @@ -205,6 +237,7 @@ async def run(self, address=3D'/tmp/qemu.socket'): await self.disconnect() =20 See `qmp.events` for more detail on event handling patterns. + """ #: Logger object used for debugging messages. logger =3D logging.getLogger(__name__) @@ -224,10 +257,12 @@ def __init__( Events.__init__(self) =20 #: Whether or not to await a greeting after establishing a connect= ion. + #: Defaults to True; QGA servers expect this to be False. self.await_greeting: bool =3D True =20 - #: Whether or not to perform capabilities negotiation upon connect= ion. - #: Implies `await_greeting`. + #: Whether or not to perform capabilities negotiation upon + #: connection. Implies `await_greeting`. Defaults to True; QGA + #: servers expect this to be False. self.negotiate: bool =3D True =20 # Cached Greeting, if one was awaited. @@ -244,7 +279,13 @@ def __init__( =20 @property def greeting(self) -> Optional[Greeting]: - """The `Greeting` from the QMP server, if any.""" + """ + The `Greeting` from the QMP server, if any. + + Defaults to ``None``, and will be set after a greeting is + received during the connection process. It is reset at the start + of each connection attempt. + """ return self._greeting =20 @upper_half @@ -385,7 +426,7 @@ async def _on_message(self, msg: Message) -> None: # This is very likely a server parsing error. # It doesn't inherently belong to any pending execution. # Instead of performing clever recovery, just terminate. - # See "NOTE" in qmp-spec.rst, section "Error". + # See "NOTE" in interop/qmp-spec, "Error" section. raise ServerParseError( ("Server sent an error response without an ID, " "but there are no ID-less executions pending. " @@ -393,7 +434,7 @@ async def _on_message(self, msg: Message) -> None: msg ) =20 - # qmp-spec.rst, section "Commands Responses": + # qmp-spec.rst, "Commands Responses" section: # 'Clients should drop all the responses # that have an unknown "id" field.' self.logger.log( @@ -566,7 +607,7 @@ async def _raw( @require(Runstate.RUNNING) async def execute_msg(self, msg: Message) -> object: """ - Execute a QMP command and return its value. + Execute a QMP command on the server and return its value. =20 :param msg: The QMP `Message` to execute. =20 @@ -578,7 +619,9 @@ async def execute_msg(self, msg: Message) -> object: If the QMP `Message` does not have either the 'execute' or 'exec-oob' fields set. :raise ExecuteError: When the server returns an error response. - :raise ExecInterruptedError: if the connection was terminated earl= y. + :raise ExecInterruptedError: + If the connection was disrupted before + receiving a reply from the server. """ if not ('execute' in msg or 'exec-oob' in msg): raise ValueError("Requires 'execute' or 'exec-oob' message") @@ -617,9 +660,11 @@ def make_execute_msg(cls, cmd: str, =20 :param cmd: QMP command name. :param arguments: Arguments (if any). Must be JSON-serializable. - :param oob: If `True`, execute "out of band". + :param oob: + If `True`, execute "out of band". See `interop/qmp-spec` + section "Out-of-band execution". =20 - :return: An executable QMP `Message`. + :return: A QMP `Message` that can be executed with `execute_msg()`. """ msg =3D Message({'exec-oob' if oob else 'execute': cmd}) if arguments is not None: @@ -631,18 +676,22 @@ async def execute(self, cmd: str, arguments: Optional[Mapping[str, object]] =3D None, oob: bool =3D False) -> object: """ - Execute a QMP command and return its value. + Execute a QMP command on the server and return its value. =20 :param cmd: QMP command name. :param arguments: Arguments (if any). Must be JSON-serializable. - :param oob: If `True`, execute "out of band". + :param oob: + If `True`, execute "out of band". See `interop/qmp-spec` + section "Out-of-band execution". =20 :return: The command execution return value from the server. The type of object returned depends on the command that was issued, though most in QEMU return a `dict`. :raise ExecuteError: When the server returns an error response. - :raise ExecInterruptedError: if the connection was terminated earl= y. + :raise ExecInterruptedError: + If the connection was disrupted before + receiving a reply from the server. """ msg =3D self.make_execute_msg(cmd, arguments, oob=3Doob) return await self.execute_msg(msg) @@ -650,8 +699,20 @@ async def execute(self, cmd: str, @upper_half @require(Runstate.RUNNING) def send_fd_scm(self, fd: int) -> None: - """ - Send a file descriptor to the remote via SCM_RIGHTS. + """Send a file descriptor to the remote via SCM_RIGHTS. + + This method does not close the file descriptor. + + :param fd: The file descriptor to send to QEMU. + + This is an advanced feature of QEMU where file descriptors can + be passed from client to server. This is usually used as a + security measure to isolate the QEMU process from being able to + open its own files. See the QMP commands ``getfd`` and + ``add-fd`` for more information. + + See `socket.socket.sendmsg` for more information on the Python + implementation for sending file descriptors over a UNIX socket. """ assert self._writer is not None sock =3D self._writer.transport.get_extra_info('socket') diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py index c923ff09e1f..f8188005685 100644 --- a/python/qemu/qmp/qmp_shell.py +++ b/python/qemu/qmp/qmp_shell.py @@ -10,9 +10,15 @@ # =20 """ -Low-level QEMU shell on top of QMP. +qmp-shell - An interactive QEMU shell powered by QMP =20 -usage: qmp-shell [-h] [-H] [-N] [-v] [-p] qmp_server +qmp-shell offers a simple shell with a convenient shorthand syntax as an +alternative to typing JSON by hand. This syntax is not standardized and +is not meant to be used as a scriptable interface. This shorthand *may* +change incompatibly in the future, and it is strongly encouraged to use +the QMP library to provide API-stable scripting when needed. + +usage: qmp-shell [-h] [-H] [-v] [-p] [-l LOGFILE] [-N] qmp_server =20 positional arguments: qmp_server < UNIX socket path | TCP address:port > @@ -20,41 +26,52 @@ optional arguments: -h, --help show this help message and exit -H, --hmp Use HMP interface - -N, --skip-negotiation - Skip negotiate (for qemu-ga) -v, --verbose Verbose (echo commands sent and received) -p, --pretty Pretty-print JSON + -l LOGFILE, --logfile LOGFILE + Save log of all QMP messages to PATH + -N, --skip-negotiation + Skip negotiate (for qemu-ga) =20 +Usage +----- =20 -Start QEMU with: +First, start QEMU with:: =20 -# qemu [...] -qmp unix:./qmp-sock,server + > qemu [...] -qmp unix:./qmp-sock,server=3Don[,wait=3Doff] =20 -Run the shell: +Then run the shell, passing the address of the socket:: =20 -$ qmp-shell ./qmp-sock + > qmp-shell ./qmp-sock =20 -Commands have the following format: +Syntax +------ =20 - < command-name > [ arg-name1=3Darg1 ] ... [ arg-nameN=3DargN ] +Commands have the following format:: =20 -For example: + < command-name > [ arg-name1=3Darg1 ] ... [ arg-nameN=3DargN ] =20 -(QEMU) device_add driver=3De1000 id=3Dnet1 -{'return': {}} -(QEMU) +For example, to add a network device:: =20 -key=3Dvalue pairs also support Python or JSON object literal subset notati= ons, -without spaces. Dictionaries/objects {} are supported as are arrays []. + (QEMU) device_add driver=3De1000 id=3Dnet1 + {'return': {}} + (QEMU) =20 - example-command arg-name1=3D{'key':'value','obj'=3D{'prop':"value"}} +key=3Dvalue pairs support either Python or JSON object literal notations, +**without spaces**. Dictionaries/objects ``{}`` are supported, as are +arrays ``[]``:: =20 -Both JSON and Python formatting should work, including both styles of -string literal quotes. Both paradigms of literal values should work, -including null/true/false for JSON and None/True/False for Python. + example-command arg-name1=3D{'key':'value','obj'=3D{'prop':"value"}} =20 +Either JSON or Python formatting for compound values works, including +both styles of string literal quotes (either single or double +quotes). Both paradigms of literal values are accepted, including +``null/true/false`` for JSON and ``None/True/False`` for Python. =20 -Transactions have the following multi-line format: +Transactions +------------ + +Transactions have the following multi-line format:: =20 transaction( action-name1 [ arg-name1=3Darg1 ] ... [arg-nameN=3DargN ] @@ -62,11 +79,11 @@ action-nameN [ arg-name1=3Darg1 ] ... [arg-nameN=3DargN ] ) =20 -One line transactions are also supported: +One line transactions are also supported:: =20 transaction( action-name1 ... ) =20 -For example: +For example:: =20 (QEMU) transaction( TRANS> block-dirty-bitmap-add node=3Ddrive0 name=3Dbitmap1 @@ -75,9 +92,35 @@ {"return": {}} (QEMU) =20 -Use the -v and -p options to activate the verbose and pretty-print options, -which will echo back the properly formatted JSON-compliant QMP that is bei= ng -sent to QEMU, which is useful for debugging and documentation generation. +Commands +-------- + +Autocomplete of command names using is supported. Pressing +at a blank CLI prompt will show you a list of all available commands +that the connected QEMU instance supports. + +For documentation on QMP commands and their arguments, please see +`qmp ref`. + +Events +------ + +qmp-shell will display events received from the server, but this version +does not do so asynchronously. To check for new events from the server, +press on a blank line:: + + (QEMU) =E2=8F=8E + {'timestamp': {'seconds': 1660071944, 'microseconds': 184667}, + 'event': 'STOP'} + +Display options +--------------- + +Use the -v and -p options to activate the verbose and pretty-print +options, which will echo back the properly formatted JSON-compliant QMP +that is being sent to QEMU. This is useful for debugging to see the +wire-level QMP data being exchanged, and generating output for use in +writing documentation for QEMU. """ =20 import argparse @@ -525,6 +568,8 @@ def common_parser() -> argparse.ArgumentParser: help=3D'Pretty-print JSON') parser.add_argument('-l', '--logfile', help=3D'Save log of all QMP messages to PATH') + # NOTE: When changing arguments, update both this module docstring + # and the manpage synopsis in docs/man/qmp_shell.rst. return parser =20 =20 @@ -567,8 +612,35 @@ def main() -> None: =20 def main_wrap() -> None: """ - qmp-shell-wrap entry point: parse command line arguments and - start the REPL. + qmp-shell-wrap - QEMU + qmp-shell launcher utility + + Launch QEMU and connect to it with `qmp-shell` in a single command. + CLI arguments will be forwarded to qemu, with additional arguments + added to allow `qmp-shell` to then connect to the recently launched + QEMU instance. + + usage: qmp-shell-wrap [-h] [-H] [-v] [-p] [-l LOGFILE] ... + + positional arguments: + command QEMU command line to invoke + + optional arguments: + -h, --help show this help message and exit + -H, --hmp Use HMP interface + -v, --verbose Verbose (echo commands sent and received) + -p, --pretty Pretty-print JSON + -l LOGFILE, --logfile LOGFILE + Save log of all QMP messages to PATH + + Usage + ----- + + Prepend "qmp-shell-wrap" to your usual QEMU command line:: + + > qmp-shell-wrap qemu-system-x86_64 -M q35 -m 4096 -display none + Welcome to the QMP low-level shell! + Connected + (QEMU) """ parser =3D common_parser() parser.add_argument('command', nargs=3Dargparse.REMAINDER, diff --git a/python/qemu/qmp/util.py b/python/qemu/qmp/util.py index 47ec39a8b5e..a8229e55245 100644 --- a/python/qemu/qmp/util.py +++ b/python/qemu/qmp/util.py @@ -49,7 +49,7 @@ def get_or_create_event_loop() -> asyncio.AbstractEventLo= op: =20 async def flush(writer: asyncio.StreamWriter) -> None: """ - Utility function to ensure a StreamWriter is *fully* drained. + Utility function to ensure an `asyncio.StreamWriter` is *fully* draine= d. =20 `asyncio.StreamWriter.drain` only promises we will return to below the "high-water mark". This function ensures we flush the entire @@ -89,7 +89,7 @@ def bottom_half(func: T) -> T: =20 These methods do not, in general, have the ability to directly report information to a caller=E2=80=99s context and will usually be - collected as a Task result instead. + collected as an `asyncio.Task` result instead. =20 They must not call upper-half functions directly. """ @@ -105,8 +105,11 @@ def exception_summary(exc: BaseException) -> str: """ Return a summary string of an arbitrary exception. =20 - It will be of the form "ExceptionType: Error Message", if the error + It will be of the form "ExceptionType: Error Message" if the error string is non-empty, and just "ExceptionType" otherwise. + + This code is based on CPython's implementation of + `traceback.TracebackException.format_exception_only`. """ name =3D type(exc).__qualname__ smod =3D type(exc).__module__ --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758550; cv=none; d=zohomail.com; s=zohoarc; b=d+CKbgu5Ls3g7I0dmwlV19q6eR4bQcX1I/adu/M3wTUcAOtFq9k52vwWQr8iNbrWkAum3u5n/TyWLvrq+0dr22D4hmNy8U5xt/0Q/sdfXdh20FUB4MiCpfgtBqr6c+VlidKqrUJjt0rwlhbRfBUeKMdzEnkKdJ7KBjgyrzBsgkQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758550; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nO2aYUg1ahQ3Mq+J7zWco6tMP/5HUaKkGg+NPQtmCCc=; b=k7pqmrjX8eD6QwEOp96StST4ED8/ztK1klP0ORHOhDrDE3Ujj7/ERSxJ8oeVg1ql6+7OLz66G+oFSyPawIwJg9gDjEPOPssFjbnKhEtlv5yOg/MrouZTLSdEA/m1Ku80odmDIXt66SdUo2XGIHiKUHSdmFSepTlLwnd0qlTtfFs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758550404331.6808337983381; Mon, 1 Sep 2025 13:29:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB8Q-0004QY-W8; Mon, 01 Sep 2025 16:28:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB81-0004LY-2r for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB7z-0000C7-FP for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:48 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-211-qiZM3qFvPuypJuT1CVaMAg-1; Mon, 01 Sep 2025 16:27:45 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7295519560AA; Mon, 1 Sep 2025 20:27:44 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F218619560B4; Mon, 1 Sep 2025 20:27:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758466; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nO2aYUg1ahQ3Mq+J7zWco6tMP/5HUaKkGg+NPQtmCCc=; b=ifreplnbsSm0DozfNMlXvfNjU5o2iQ5yRLxsPUAjpj+1AO4Mh5rUpxUya3KAdCx0aZ/MS5 po3VE6amR/ulE6N0vKTgQ2MCtfB7YUiDdczwZ+S3CI3gYH24dn3px3LL6Ys87u1UNQ1a8l rScLmh/oYPvEmrNQTRuSr17Odb8GP4Q= X-MC-Unique: qiZM3qFvPuypJuT1CVaMAg-1 X-Mimecast-MFC-AGG-ID: qiZM3qFvPuypJuT1CVaMAg_1756758464 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 15/19] iotests: drop compat for old version context manager Date: Mon, 1 Sep 2025 16:26:57 -0400 Message-ID: <20250901202702.2971212-16-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758553437124100 From: Daniel P. Berrang=C3=A9 Our minimum python is now 3.9, so back compat with prior python versions is no longer required. Signed-off-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py | 7 ++----- tests/qemu-iotests/testrunner.py | 9 ++------- 2 files changed, 4 insertions(+), 12 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 6326e46b7b1..29caaa8a349 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -22,15 +22,12 @@ from pathlib import Path import shutil import collections +import contextlib import random import subprocess import glob from typing import List, Dict, Any, Optional =20 -if sys.version_info >=3D (3, 9): - from contextlib import AbstractContextManager as ContextManager -else: - from typing import ContextManager =20 DEF_GDB_OPTIONS =3D 'localhost:12345' =20 @@ -58,7 +55,7 @@ def get_default_machine(qemu_prog: str) -> str: return default_machine =20 =20 -class TestEnv(ContextManager['TestEnv']): +class TestEnv(contextlib.AbstractContextManager['TestEnv']): """ Manage system environment for running tests =20 diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunn= er.py index 2e236c8fa39..14cc8492f9f 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -30,11 +30,6 @@ from typing import List, Optional, Any, Sequence, Dict from testenv import TestEnv =20 -if sys.version_info >=3D (3, 9): - from contextlib import AbstractContextManager as ContextManager -else: - from typing import ContextManager - =20 def silent_unlink(path: Path) -> None: try: @@ -57,7 +52,7 @@ def file_diff(file1: str, file2: str) -> List[str]: return res =20 =20 -class LastElapsedTime(ContextManager['LastElapsedTime']): +class LastElapsedTime(contextlib.AbstractContextManager['LastElapsedTime']= ): """ Cache for elapsed time for tests, to show it during new test run =20 It is safe to use get() at any time. To use update(), you must either @@ -112,7 +107,7 @@ def __init__(self, status: str, description: str =3D '', self.interrupted =3D interrupted =20 =20 -class TestRunner(ContextManager['TestRunner']): +class TestRunner(contextlib.AbstractContextManager['TestRunner']): shared_self =3D None =20 @staticmethod --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758689; cv=none; d=zohomail.com; s=zohoarc; b=QPELTBiP93qRV4lMAGPbOTb21hn1p1B+TyonP4YtlSVW7ROnUt9srUvMULZoaV7tCD5cpia8rAwgkYYXxCAITbaU3NGjcmmF5FETfG9Pru49YApOPyQthdoWHoG8w0R0RCLSfw1JUK/s/vu2oqsPwk2NURqrcNkxzoFIStOdNWU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758689; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/BmZ9w7l2CougITuHQBHRbvcKQikdf1OA1lcFK/juXY=; b=dmXPXC+qjc+aVqNkgufq4NBFat5DC/26HrDfKd5wI03mBCgUh+AHZGRP+67bmaSPwKirAmcfs058v0BMLA+hIQNCYlU9uVpwJ4eikIZ1S3PbE03gn2HAC68sKFlwPB6Mekk7JXwlpnf1K86nERqPLD/R4dyg+wzh1Qi2TFf1r7g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758689547309.33750025145844; Mon, 1 Sep 2025 13:31:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB9I-0005MA-8i; Mon, 01 Sep 2025 16:29:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB8E-0004Qa-Qh for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:28:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB8B-0000Ev-5l for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:28:01 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-45-VdTjywabNmeiY1J3u4zuFA-1; Mon, 01 Sep 2025 16:27:50 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6265518003FD; Mon, 1 Sep 2025 20:27:47 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ECCAE19560AB; Mon, 1 Sep 2025 20:27:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758477; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/BmZ9w7l2CougITuHQBHRbvcKQikdf1OA1lcFK/juXY=; b=SgWlAr4542JnAEkD/JTTqOk55xRIp7oWGWF5Ku3Tl97GqlmVq9Qz7irRtMaTno683lllQR g7rOqdDbpXnvhIQL8yRmYHTL9pYwFReoYEpTudUl35OOq906HL+GsWz1NFnUzJ9dXMX6cF COFROjDBnJgX3CivD32JQAzqyYLRmyU= X-MC-Unique: VdTjywabNmeiY1J3u4zuFA-1 X-Mimecast-MFC-AGG-ID: VdTjywabNmeiY1J3u4zuFA_1756758467 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 16/19] python: ensure QEMUQtestProtocol closes its socket Date: Mon, 1 Sep 2025 16:26:58 -0400 Message-ID: <20250901202702.2971212-17-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758690830124100 From: Daniel P. Berrang=C3=A9 While QEMUQtestMachine closes the socket that was passed to QEMUQtestProtocol, the python resource leak manager still believes that the copy QEMUQtestProtocol holds is open. We must explicitly call close to avoid this leak warnnig. Signed-off-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow --- python/qemu/machine/qtest.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py index 4f5ede85b23..781f674ffaf 100644 --- a/python/qemu/machine/qtest.py +++ b/python/qemu/machine/qtest.py @@ -177,6 +177,8 @@ def _post_shutdown(self) -> None: self._qtest_sock_pair[0].close() self._qtest_sock_pair[1].close() self._qtest_sock_pair =3D None + if self._qtest is not None: + self._qtest.close() super()._post_shutdown() =20 def qtest(self, cmd: str) -> str: --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758605; cv=none; d=zohomail.com; s=zohoarc; b=T50Sjbo9XZZNZRwavw+5r4jlny4/oPceSN1hU5OmoCKR+O10boyFyGnmOZzHBBHjbZxOyya83xfDSwjokq3h4jibzpzvwD823lXoNmewHN0srw0AIyDPxaZ3+EH4s1me2qGuY7CJvMWp/iKuTZab1MwpktQej6v+74RNVDdHflY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758605; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=redY3J6Go7FDZCUZVMBmRGiuKOFKplF9JXDgVj4SBW8=; b=nuzo44JIzZeHO6Jnjz4Hh5P/2Pp43D3JNojX4sMiBKML3MTtj5IEULakAwm1ym7xlcS0JkNIN68cqyFaEiupjTZ6hm4oFkNUjR1nayx/crMpSqOChfuxZyk23l66ORtZ6ygAcRM8g+oHQ490LOWkdIPgKP98BJA6aiucO/y7oMQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17567586054742.6057611569797245; Mon, 1 Sep 2025 13:30:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB9H-0005Jp-Qd; Mon, 01 Sep 2025 16:29:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB89-0004O3-Bf for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:28:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB86-0000E8-Rg for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:55 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-686-Yfl1PKl2NI2ep4y1pXwSXw-1; Mon, 01 Sep 2025 16:27:50 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C004E180034B; Mon, 1 Sep 2025 20:27:49 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A6D8F19560AB; Mon, 1 Sep 2025 20:27:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758474; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=redY3J6Go7FDZCUZVMBmRGiuKOFKplF9JXDgVj4SBW8=; b=DGbLMlNgcOqGicWtisYbdflU9verBneUxBJENs10M/hq1fwKjBd2fDvGrJTFt1dKeDxyzH MQDzOBDVFbnR0QsEVup618sVRvqaQaAFrMqAOFIfdtgcBROyK+zYScmjDD5fTzekVNforz 5ddOtNUsFoZU0ezJry5njzdC0Y5C5eo= X-MC-Unique: Yfl1PKl2NI2ep4y1pXwSXw-1 X-Mimecast-MFC-AGG-ID: Yfl1PKl2NI2ep4y1pXwSXw_1756758469 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 17/19] iotests/147: ensure temporary sockets are closed before exiting Date: Mon, 1 Sep 2025 16:26:59 -0400 Message-ID: <20250901202702.2971212-18-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758607832124100 From: Daniel P. Berrang=C3=A9 This avoids the python resource leak detector from issuing warnings in the iotests. Signed-off-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow --- tests/qemu-iotests/147 | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 index 6d6f077a14d..3e14bd389a4 100755 --- a/tests/qemu-iotests/147 +++ b/tests/qemu-iotests/147 @@ -277,6 +277,7 @@ class BuiltinNBD(NBDBlockdevAddBase): } } self.client_test(filename, flatten_sock_addr(address), 'nbd-export= ') =20 + sockfd.close() self._server_down() =20 =20 --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758548; cv=none; d=zohomail.com; s=zohoarc; b=Q2UuNllj5pHnaYoBb7tkKzfKJnKgeH3L2EgTEtUn03Kq93KOtwGjkw6m9r9fYRTqwDTuEGVGDDbWeSFD46+O02mZfcubm8GjxiCPcSzxcm3pMR6IoBeEfMaqP/NxZ5SxATGNGLPYmKCpVJ1+QIWNgRRnmZKiQriH496jXL0Zuao= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758548; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=c/lFu6o8E1HCKXvUoJO2x7slIVPWYpyPrHnksOMI3CY=; b=U7qszhVuDfm4QVrBslgTWgbbXhvwN9HuA1MneO/bNMujybdVd1kJU1PDGO5E/QLJJ5f2wZtVJsIV5LGnGBbgEWJ7SMSRu/9Qg2P+V3LOTmuueUlBKPfQ3Z7HVztExdnX0bN08WVAL8+oIH5lDswzc09IUAHFfl9n34mVaIcIlns= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756758548242516.1683509753221; Mon, 1 Sep 2025 13:29:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB9E-00054R-2j; Mon, 01 Sep 2025 16:29:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB8B-0004Ot-Dl for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:28:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB89-0000Ea-7r for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:27:58 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-610-UGIdFZkXND2J3--x5b-KfQ-1; Mon, 01 Sep 2025 16:27:53 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8BBAA180044F; Mon, 1 Sep 2025 20:27:52 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0F37B19560AB; Mon, 1 Sep 2025 20:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c/lFu6o8E1HCKXvUoJO2x7slIVPWYpyPrHnksOMI3CY=; b=T0Rk9gzSqEoJVKJew2sNeAFPZgEmJ2tpOX3Px/Us5lp0rn8Hr9QDoK3eogFZmwp4dcJ7eu Q2Ahc3gXD5huOR+3B+7tmHIYpLuug2+RVd48UQmDZn+c4bF1QS62jxIn8yTo/mlPsC7wqs RdMiAYR6jlWYJtCAc5g03g79uXOUFJI= X-MC-Unique: UGIdFZkXND2J3--x5b-KfQ-1 X-Mimecast-MFC-AGG-ID: UGIdFZkXND2J3--x5b-KfQ_1756758472 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 18/19] iotests/151: ensure subprocesses are cleaned up Date: Mon, 1 Sep 2025 16:27:00 -0400 Message-ID: <20250901202702.2971212-19-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758549003116600 From: Daniel P. Berrang=C3=A9 The iotest 151 creates a bunch of subprocesses, with their stdout connected to a pipe but never reads any data from them and does not gurantee the processes are killed on cleanup. This triggers resource leak warnings from python when the subprocess.Popen object is garbage collected. Signed-off-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow --- tests/qemu-iotests/151 | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 index f2ff9c5dac2..06ee3585db9 100755 --- a/tests/qemu-iotests/151 +++ b/tests/qemu-iotests/151 @@ -263,6 +263,11 @@ class TestThrottledWithNbdExportBase(iotests.QMPTestCa= se): break except subprocess.TimeoutExpired: self.vm.qtest(f'clock_step {1 * 1000 * 1000 * 1000= }') + try: + p.kill() + p.stdout.close() + except: + pass except IndexError: pass =20 --=20 2.50.1 From nobody Sun Sep 28 15:29:05 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1756758578; cv=none; d=zohomail.com; s=zohoarc; b=ZURGd/ZwpaSXJX0tTf5Y3v6l2I14M608II3Bg/apr6X95OItHOUikuValNqsB+lLg8YacVdv+KN/GN87hmJl87RY1JR5FjVcf2SVPTpjqsPrUrCHuocnCoJNcuxew7d8gYnvLWDnZy8fdAdIH772iEAhGjRBflGoLfQg05A0iOk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756758578; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=A4xBFXb45hM+ZJ7Arv4NuY5r0toY1PnfWxR1gk8ah2U=; b=GNuh0k0apfv9E0soU4qMTIrvD5bNrGDF1Q4x7hgqmWAlcxp5Va0Ezgd8rRgsIgVh9TkNyv3tWBHjVR4hJWiy7K/PYUOhtaHs0sEqABgSM4MmqWJr/cIHd/G3CUhckKpIQyHPl/fOCho7j+8EmN2KA3VTV+1rcXiscWzT/b6mNiI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17567585781035.193667596598516; Mon, 1 Sep 2025 13:29:38 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1utB9J-0005WJ-9f; Mon, 01 Sep 2025 16:29:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB8C-0004QN-R5 for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:28:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1utB8B-0000Et-5O for qemu-devel@nongnu.org; Mon, 01 Sep 2025 16:28:00 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-439--JuF5t1LMKaGV_uxSn7MNA-1; Mon, 01 Sep 2025 16:27:56 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6E54919560AF; Mon, 1 Sep 2025 20:27:55 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.80.85]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D032219560B4; Mon, 1 Sep 2025 20:27:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1756758477; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A4xBFXb45hM+ZJ7Arv4NuY5r0toY1PnfWxR1gk8ah2U=; b=I0BQLoSMPFGTaLGACHq+G1+q6lEpaQq/pzEMm1IonXVyBDsYf6ww3m3D1JKvfN+q/c2DsE JXUD7TwUaGpUXBoBhZAgvjvO3wwELIRLgSFIC/540v/F97d4mRm164NlFWmWlRKN3bTw4H 8faD+b4bGk4skld4eppbmQDAoaEozSI= X-MC-Unique: -JuF5t1LMKaGV_uxSn7MNA-1 X-Mimecast-MFC-AGG-ID: -JuF5t1LMKaGV_uxSn7MNA_1756758475 From: John Snow To: qemu-devel@nongnu.org Cc: Hanna Reitz , Cleber Rosa , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Daniel=20Berrang=C3=A9?= , Kevin Wolf Subject: [PATCH 19/19] iotests/check: always enable all python warnings Date: Mon, 1 Sep 2025 16:27:01 -0400 Message-ID: <20250901202702.2971212-20-jsnow@redhat.com> In-Reply-To: <20250901202702.2971212-1-jsnow@redhat.com> References: <20250901202702.2971212-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1756758579328116600 From: Daniel P. Berrang=C3=A9 Of most importance is that this gives us a heads-up if anything we rely on has been deprecated. The default python behaviour only emits a warning if triggered from __main__ which is very limited. Setting the env variable further ensures that any python child processes will also display warnings. Signed-off-by: Daniel P. Berrang=C3=A9 Signed-off-by: John Snow --- tests/qemu-iotests/check | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 545f9ec7bdd..d9b7c1d5989 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -21,6 +21,7 @@ import sys import argparse import shutil from pathlib import Path +import warnings =20 from findtests import TestFinder from testenv import TestEnv @@ -137,6 +138,9 @@ def make_argparser() -> argparse.ArgumentParser: =20 =20 if __name__ =3D=3D '__main__': + warnings.simplefilter("default") + os.environ["PYTHONWARNINGS"] =3D "default" + args =3D make_argparser().parse_args() =20 env =3D TestEnv(source_dir=3Dargs.source_dir, --=20 2.50.1