From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359661398205.84740550364586; Thu, 29 Mar 2018 14:41:01 -0700 (PDT) Received: from localhost ([::1]:60141 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fI4-0002oi-DV for importer@patchew.org; Thu, 29 Mar 2018 17:41:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60919) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGL-0001YB-M5 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGK-0007li-Vi for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55052) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGK-0007iz-Qd for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:12 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 205A5C015E9A; Thu, 29 Mar 2018 21:39:12 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id E746360928; Thu, 29 Mar 2018 21:39:05 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:40 -0300 Message-Id: <20180329213857.15499-2-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 29 Mar 2018 21:39:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 01/18] qmp.py: Make it safe to call close() any time X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will allow us to simplify the error handling and shutdown logic in qemu.py. Signed-off-by: Eduardo Habkost --- scripts/qmp/qmp.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py index 5c8cf6a056..078ce65f3b 100644 --- a/scripts/qmp/qmp.py +++ b/scripts/qmp/qmp.py @@ -244,7 +244,9 @@ class QEMUMonitorProtocol(object): =20 def close(self): self.__sock.close() - self.__sockfile.close() + if self.__sockfile is not None: + self.__sockfile.close() + self.__sockfile =3D None =20 def settimeout(self, timeout): self.__sock.settimeout(timeout) --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359815795513.4892758223631; Thu, 29 Mar 2018 14:43:35 -0700 (PDT) Received: from localhost ([::1]:60169 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fKY-0005CJ-UF for importer@patchew.org; Thu, 29 Mar 2018 17:43:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60954) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGP-0001b6-37 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGM-0007vB-SO for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51560) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGM-0007rf-MY for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:14 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E5FBE7F7C6; Thu, 29 Mar 2018 21:39:13 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 79A5B5C8BF; Thu, 29 Mar 2018 21:39:13 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:41 -0300 Message-Id: <20180329213857.15499-3-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 29 Mar 2018 21:39:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 02/18] qmp.py: Fix error handling for Python 3 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" socket.error doesn't behave like a tuple in Python 3, but we can use error.args on both Python 2.7 and 3. Signed-off-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- scripts/qmp/qmp.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py index 078ce65f3b..e9cb6b2683 100644 --- a/scripts/qmp/qmp.py +++ b/scripts/qmp/qmp.py @@ -108,7 +108,7 @@ class QEMUMonitorProtocol(object): try: self.__json_read() except socket.error as err: - if err[0] =3D=3D errno.EAGAIN: + if err.args[0] =3D=3D errno.EAGAIN: # No data available pass self.__sock.setblocking(1) @@ -168,9 +168,9 @@ class QEMUMonitorProtocol(object): try: self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) except socket.error as err: - if err[0] =3D=3D errno.EPIPE: + if err.args[0] =3D=3D errno.EPIPE: return - raise socket.error(err) + raise resp =3D self.__json_read() self.logger.debug("<<< %s", resp) return resp --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360001095257.7974653627808; Thu, 29 Mar 2018 14:46:41 -0700 (PDT) Received: from localhost ([::1]:60207 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fNN-000834-51 for importer@patchew.org; Thu, 29 Mar 2018 17:46:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60987) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGV-0001ha-Lp for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGU-0008M4-QX for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56102) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGU-0008Ja-Kr for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:22 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EAC4C2F30C3; Thu, 29 Mar 2018 21:39:21 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C73460BEC; Thu, 29 Mar 2018 21:39:15 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:42 -0300 Message-Id: <20180329213857.15499-4-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 29 Mar 2018 21:39:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 03/18] qmp.py: Cleanly handle unexpectedly closed socket X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" QEMUMonitorProtocol.cmd() returns None if the socket was closed, so callers must handle this case explicltly. Signed-off-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- scripts/qmp/qmp.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py index e9cb6b2683..91b537ea04 100644 --- a/scripts/qmp/qmp.py +++ b/scripts/qmp/qmp.py @@ -73,6 +73,8 @@ class QEMUMonitorProtocol(object): raise QMPConnectError # Greeting seems ok, negotiate capabilities resp =3D self.cmd('qmp_capabilities') + if resp is None: + raise QMPConnectError("QMP connection unexpectedly closed") if "return" in resp: return greeting raise QMPCapabilitiesError @@ -182,6 +184,8 @@ class QEMUMonitorProtocol(object): @param name: command name (string) @param args: command arguments (dict) @param cmd_id: command id (dict, list, string or int) + @return QMP response as a Python dict or None if the connection has + been closed """ qmp_cmd =3D {'execute': name} if args: @@ -195,6 +199,8 @@ class QEMUMonitorProtocol(object): Build and send a QMP command to the monitor, report errors if any """ ret =3D self.cmd(cmd, kwds) + if ret is None: + raise QMPConnectError("QMP connection unexpectedly closed") if "error" in ret: raise Exception(ret['error']['desc']) return ret['return'] --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359835622204.51732594982275; Thu, 29 Mar 2018 14:43:55 -0700 (PDT) Received: from localhost ([::1]:60170 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fKs-0005VZ-Q8 for importer@patchew.org; Thu, 29 Mar 2018 17:43:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32780) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGe-0001sJ-1X for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGd-0000AG-7j for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43810) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGd-00009G-1l for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5589385359; Thu, 29 Mar 2018 21:39:30 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5410265AF2; Thu, 29 Mar 2018 21:39:23 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:43 -0300 Message-Id: <20180329213857.15499-5-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 29 Mar 2018 21:39:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 04/18] qemu.py: Make _vm_monitor a method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use a function instead of a data field. Less state to keep track of, less chance of bugs. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 08a3e9af5a..c051c4525a 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -76,7 +76,6 @@ class QEMUMachine(object): name =3D "qemu-%d" % os.getpid() self._name =3D name self._monitor_address =3D monitor_address - self._vm_monitor =3D None self._qemu_log_path =3D None self._qemu_log_file =3D None self._popen =3D None @@ -168,29 +167,28 @@ class QEMUMachine(object): with open(self._qemu_log_path, "r") as iolog: self._iolog =3D iolog.read() =20 + def _vm_monitor(self): + if self._monitor_address is not None: + return self._monitor_address + else: + return os.path.join(self._temp_dir, self._name + "-monitor.soc= k") + def _base_args(self): - if isinstance(self._monitor_address, tuple): - moncdev =3D "socket,id=3Dmon,host=3D%s,port=3D%s" % ( - self._monitor_address[0], - self._monitor_address[1]) + addr =3D self._vm_monitor() + if isinstance(addr, tuple): + moncdev =3D "socket,id=3Dmon,host=3D%s,port=3D%s" % (addr[0], = addr[1]) else: - moncdev =3D 'socket,id=3Dmon,path=3D%s' % self._vm_monitor + moncdev =3D 'socket,id=3Dmon,path=3D%s' % (addr) return ['-chardev', moncdev, '-mon', 'chardev=3Dmon,mode=3Dcontrol', '-display', 'none', '-vga', 'none'] =20 def _pre_launch(self): self._temp_dir =3D tempfile.mkdtemp(dir=3Dself._test_dir) - if self._monitor_address is not None: - self._vm_monitor =3D self._monitor_address - else: - self._vm_monitor =3D os.path.join(self._temp_dir, - self._name + "-monitor.sock") self._qemu_log_path =3D os.path.join(self._temp_dir, self._name + = ".log") self._qemu_log_file =3D open(self._qemu_log_path, 'wb') =20 - self._qmp =3D qmp.qmp.QEMUMonitorProtocol(self._vm_monitor, - server=3DTrue) + self._qmp =3D qmp.qmp.QEMUMonitorProtocol(self._vm_monitor(), serv= er=3DTrue) =20 def _post_launch(self): self._qmp.accept() --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360011895785.7999073948033; Thu, 29 Mar 2018 14:46:51 -0700 (PDT) Received: from localhost ([::1]:60208 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fNj-0008Jt-3e for importer@patchew.org; Thu, 29 Mar 2018 17:46:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32807) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGl-0001zO-NY for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGk-0000ca-PS for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57380) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGk-0000YU-K0 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:38 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DC92FC04BE24; Thu, 29 Mar 2018 21:39:37 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id B5BC1601A9; Thu, 29 Mar 2018 21:39:31 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:44 -0300 Message-Id: <20180329213857.15499-6-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 29 Mar 2018 21:39:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 05/18] qemu.py: Split _base_args() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Split it into _monitor_args() and _display_args(), so we can make monitor args optional later. Signed-off-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- scripts/qemu.py | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index c051c4525a..d9f85bb153 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -173,15 +173,20 @@ class QEMUMachine(object): else: return os.path.join(self._temp_dir, self._name + "-monitor.soc= k") =20 - def _base_args(self): + def _monitor_args(self): addr =3D self._vm_monitor() if isinstance(addr, tuple): moncdev =3D "socket,id=3Dmon,host=3D%s,port=3D%s" % (addr[0], = addr[1]) else: moncdev =3D 'socket,id=3Dmon,path=3D%s' % (addr) return ['-chardev', moncdev, - '-mon', 'chardev=3Dmon,mode=3Dcontrol', - '-display', 'none', '-vga', 'none'] + '-mon', 'chardev=3Dmon,mode=3Dcontrol'] + + def _display_args(self): + return ['-display', 'none', '-vga', 'none'] + + def _base_args(self): + return self._monitor_args() + self._display_args() =20 def _pre_launch(self): self._temp_dir =3D tempfile.mkdtemp(dir=3Dself._test_dir) --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359701455584.6493352749742; Thu, 29 Mar 2018 14:41:41 -0700 (PDT) Received: from localhost ([::1]:60142 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fIi-0003cZ-Fp for importer@patchew.org; Thu, 29 Mar 2018 17:41:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32821) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGn-00023w-Nn for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGm-0000kB-KX for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43898) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGm-0000hI-F9 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:40 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B84998124D; Thu, 29 Mar 2018 21:39:39 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4561D5C8BF; Thu, 29 Mar 2018 21:39:39 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:45 -0300 Message-Id: <20180329213857.15499-7-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 29 Mar 2018 21:39:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 06/18] qemu.py: Move _load_io_log() call to _post_shutdown() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" All callers of _post_shutdown() call _load_io_log(), so it's easier to simply call it inside _post_shutdown(). Signed-off-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- scripts/qemu.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index d9f85bb153..84bb3da613 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -199,6 +199,7 @@ class QEMUMachine(object): self._qmp.accept() =20 def _post_shutdown(self): + self._load_io_log() if self._qemu_log_file is not None: self._qemu_log_file.close() self._qemu_log_file =3D None @@ -250,7 +251,6 @@ class QEMUMachine(object): '''Wait for the VM to power off''' self._popen.wait() self._qmp.close() - self._load_io_log() self._post_shutdown() =20 def shutdown(self): @@ -263,7 +263,6 @@ class QEMUMachine(object): self._popen.kill() self._popen.wait() =20 - self._load_io_log() self._post_shutdown() =20 exitcode =3D self.exitcode() --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359877963604.7304728601446; Thu, 29 Mar 2018 14:44:37 -0700 (PDT) Received: from localhost ([::1]:60171 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fLT-00064T-D7 for importer@patchew.org; Thu, 29 Mar 2018 17:44:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32867) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fGt-0002BJ-RX for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fGt-00014R-1t for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57166) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fGs-000132-Rg for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2339BC036757; Thu, 29 Mar 2018 21:39:46 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B6C65D9CA; Thu, 29 Mar 2018 21:39:40 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:46 -0300 Message-Id: <20180329213857.15499-8-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 29 Mar 2018 21:39:46 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 07/18] qemu.py: Use wait() logic inside shutdown() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This way we will have a single method where we wait for the QEMU process to finish. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 84bb3da613..4a71542c6b 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -258,12 +258,10 @@ class QEMUMachine(object): if self.is_running(): try: self._qmp.cmd('quit') - self._qmp.close() except: self._popen.kill() - self._popen.wait() =20 - self._post_shutdown() + self.wait() =20 exitcode =3D self.exitcode() if exitcode is not None and exitcode < 0: --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360176405446.43308346514414; Thu, 29 Mar 2018 14:49:36 -0700 (PDT) Received: from localhost ([::1]:60234 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fQN-0002ST-Eh for importer@patchew.org; Thu, 29 Mar 2018 17:49:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32905) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fH1-0002Ix-7a for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fH0-0001Bm-Dh for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56332) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fH0-0001Am-8f for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:39:54 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8D6B818E3CE; Thu, 29 Mar 2018 21:39:53 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 72DFC65AC5; Thu, 29 Mar 2018 21:39:47 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:47 -0300 Message-Id: <20180329213857.15499-9-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 29 Mar 2018 21:39:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 08/18] qemu.py: Close _qmp inside _post_shutdown() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This way all shutdown-related cleanup is kept in a single place. While at it, set _qmp to None after closing the socket, to avoid trying to reuse it by mistake later. Signed-off-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- scripts/qemu.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 4a71542c6b..1241e7f10b 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -199,6 +199,10 @@ class QEMUMachine(object): self._qmp.accept() =20 def _post_shutdown(self): + if self._qmp is not None: + self._qmp.close() + self._qmp =3D None + self._load_io_log() if self._qemu_log_file is not None: self._qemu_log_file.close() @@ -250,7 +254,6 @@ class QEMUMachine(object): def wait(self): '''Wait for the VM to power off''' self._popen.wait() - self._qmp.close() self._post_shutdown() =20 def shutdown(self): --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359729123720.1029768850816; Thu, 29 Mar 2018 14:42:09 -0700 (PDT) Received: from localhost ([::1]:60143 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fJA-0003zD-6P for importer@patchew.org; Thu, 29 Mar 2018 17:42:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32959) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHA-0002Qx-Nj for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fH8-0001Ne-V4 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57850) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fH8-0001Mx-Mk for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:02 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E26A528209; Thu, 29 Mar 2018 21:40:01 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58560601AE; Thu, 29 Mar 2018 21:39:55 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:48 -0300 Message-Id: <20180329213857.15499-10-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 29 Mar 2018 21:40:01 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 09/18] qemu.py: Make monitor optional X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will allow us to use the QEMUMachine class on test cases that don't use QMP at all. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 1241e7f10b..e82540a235 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -54,7 +54,7 @@ class QEMUMachine(object): ''' =20 def __init__(self, binary, args=3DNone, wrapper=3DNone, name=3DNone, - test_dir=3D"/var/tmp", monitor_address=3DNone, + test_dir=3D"/var/tmp", monitor_address=3DNone, qmp=3DTrue, socket_scm_helper=3DNone): ''' Initialize a QEMUMachine @@ -65,6 +65,7 @@ class QEMUMachine(object): @param name: prefix for socket and log file names (default: qemu-P= ID) @param test_dir: where to create socket and log file @param monitor_address: address for QMP monitor + @param qmp: if False, disable QMP monitor @param socket_scm_helper: helper program, required for send_fd_scm= ()" @note: Qemu process is not started until launch() is used. ''' @@ -90,6 +91,7 @@ class QEMUMachine(object): self._test_dir =3D test_dir self._temp_dir =3D None self._launched =3D False + self._qmp_enabled =3D qmp =20 # just in case logging wasn't configured by the main script: logging.basicConfig() @@ -168,13 +170,18 @@ class QEMUMachine(object): self._iolog =3D iolog.read() =20 def _vm_monitor(self): - if self._monitor_address is not None: + if not self._qmp_enabled: + return None + elif self._monitor_address is not None: return self._monitor_address else: return os.path.join(self._temp_dir, self._name + "-monitor.soc= k") =20 def _monitor_args(self): addr =3D self._vm_monitor() + if addr is None: + return [] + if isinstance(addr, tuple): moncdev =3D "socket,id=3Dmon,host=3D%s,port=3D%s" % (addr[0], = addr[1]) else: @@ -193,10 +200,14 @@ class QEMUMachine(object): self._qemu_log_path =3D os.path.join(self._temp_dir, self._name + = ".log") self._qemu_log_file =3D open(self._qemu_log_path, 'wb') =20 - self._qmp =3D qmp.qmp.QEMUMonitorProtocol(self._vm_monitor(), serv= er=3DTrue) + self._qmp =3D \ + qmp.qmp.QEMUMonitorProtocol(self._vm_monitor(), server=3DTrue)= \ + if self._qmp_enabled \ + else None =20 def _post_launch(self): - self._qmp.accept() + if self._qmp: + self._qmp.accept() =20 def _post_shutdown(self): if self._qmp is not None: --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360372165127.26450045598438; Thu, 29 Mar 2018 14:52:52 -0700 (PDT) Received: from localhost ([::1]:60258 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fTO-0004t0-84 for importer@patchew.org; Thu, 29 Mar 2018 17:52:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32973) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHC-0002SL-Db for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHB-0001S1-Mw for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57656) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHB-0001QX-Ep for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:05 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B66374903F; Thu, 29 Mar 2018 21:40:04 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 12FF765AC3; Thu, 29 Mar 2018 21:40:03 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:49 -0300 Message-Id: <20180329213857.15499-11-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 29 Mar 2018 21:40:04 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 10/18] qemu.py: Set _launched = False on _post_shutdown X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will allow a sequence like: vm.launch() vm.cmd('quit') vm.wait() # triggers post-shutdown code vm.launch() to work. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index e82540a235..226d2c4d48 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -225,6 +225,8 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir =3D None =20 + self._launched =3D False + def launch(self): """ Launch the VM and make sure we cleanup and expose the @@ -286,8 +288,6 @@ class QEMUMachine(object): command =3D '' LOG.warn(msg, exitcode, command) =20 - self._launched =3D False - def qmp(self, cmd, conv_keys=3DTrue, **args): '''Invoke a QMP command and return the response dict''' qmp_args =3D dict() --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360198805566.9342259442464; Thu, 29 Mar 2018 14:49:58 -0700 (PDT) Received: from localhost ([::1]:60236 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fQj-0002lZ-Tk for importer@patchew.org; Thu, 29 Mar 2018 17:49:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33043) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHN-0002hR-Di for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHK-0001mJ-C5 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57694) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHJ-0001j5-Qo for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:14 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1F73268102; Thu, 29 Mar 2018 21:40:13 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id CBF6A60928; Thu, 29 Mar 2018 21:40:06 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:50 -0300 Message-Id: <20180329213857.15499-12-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 29 Mar 2018 21:40:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 11/18] qemu.py: Log crashes inside _post_shutdown() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will allow us to log QEMU crashes even if the test code uses .wait() instead of .shutdown(). Signed-off-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- scripts/qemu.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 226d2c4d48..e19e4b34d0 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -225,6 +225,15 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir =3D None =20 + exitcode =3D self.exitcode() + if exitcode is not None and exitcode < 0: + msg =3D 'qemu received signal %i: %s' + if self._qemu_full_args: + command =3D ' '.join(self._qemu_full_args) + else: + command =3D '' + LOG.warn(msg, exitcode, command) + self._launched =3D False =20 def launch(self): @@ -279,15 +288,6 @@ class QEMUMachine(object): =20 self.wait() =20 - exitcode =3D self.exitcode() - if exitcode is not None and exitcode < 0: - msg =3D 'qemu received signal %i: %s' - if self._qemu_full_args: - command =3D ' '.join(self._qemu_full_args) - else: - command =3D '' - LOG.warn(msg, exitcode, command) - def qmp(self, cmd, conv_keys=3DTrue, **args): '''Invoke a QMP command and return the response dict''' qmp_args =3D dict() --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522359896464376.6250634125082; Thu, 29 Mar 2018 14:44:56 -0700 (PDT) Received: from localhost ([::1]:60174 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fLr-0006Rt-GV for importer@patchew.org; Thu, 29 Mar 2018 17:44:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33094) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHW-0002p4-JN for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHT-00026U-3U for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47383) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHS-00025C-R5 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:22 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2223C19CF96; Thu, 29 Mar 2018 21:40:22 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 441045D9CA; Thu, 29 Mar 2018 21:40:14 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:51 -0300 Message-Id: <20180329213857.15499-13-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 29 Mar 2018 21:40:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 12/18] qemu.py: Only wait for process if it's still running X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" TODO: explain why Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index e19e4b34d0..aaba04b3c1 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -275,7 +275,8 @@ class QEMUMachine(object): =20 def wait(self): '''Wait for the VM to power off''' - self._popen.wait() + if self.is_running(): + self._popen.wait() self._post_shutdown() =20 def shutdown(self): --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360384112532.2493138537811; Thu, 29 Mar 2018 14:53:04 -0700 (PDT) Received: from localhost ([::1]:60474 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fTj-0005TW-Ae for importer@patchew.org; Thu, 29 Mar 2018 17:53:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33097) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHW-0002pE-Nz for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHV-0002AL-SO for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44366) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHV-00029T-JT for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:25 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C8A7D80E4A; Thu, 29 Mar 2018 21:40:24 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4F1F860928; Thu, 29 Mar 2018 21:40:24 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:52 -0300 Message-Id: <20180329213857.15499-14-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 29 Mar 2018 21:40:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 13/18] qemu.py: 'force' parameter on shutdown() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will allow test code to shut down the VM without trying to run a 'quit' QMP command. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index aaba04b3c1..045dca5d02 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -279,13 +279,24 @@ class QEMUMachine(object): self._popen.wait() self._post_shutdown() =20 - def shutdown(self): - '''Terminate the VM and clean up''' - if self.is_running(): + def shutdown(self, force=3DNone): + '''Terminate the VM and clean up + + @param force: If True, terminate QEMU process, if False always try + clean shutdown, if None terminate QEMU process if + clean shutdown fails. + ''' + if not force and self.is_running(): try: self._qmp.cmd('quit') except: - self._popen.kill() + if force is None: + force =3D True + else: + raise + + if force and self.is_running(): + self._popen.kill() =20 self.wait() =20 --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360082417283.15354257451827; Thu, 29 Mar 2018 14:48:02 -0700 (PDT) Received: from localhost ([::1]:60211 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fOr-0000oO-Hy for importer@patchew.org; Thu, 29 Mar 2018 17:48:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33143) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHb-0002uv-6G for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHa-0002Hu-4D for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52022) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHZ-0002FT-VF for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:30 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 390017F7B9; Thu, 29 Mar 2018 21:40:29 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25F09600C9; Thu, 29 Mar 2018 21:40:25 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:53 -0300 Message-Id: <20180329213857.15499-15-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 29 Mar 2018 21:40:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 14/18] qemu.py: Don't try to quit cleanly on exceptions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Exceptions when starting a VM probably mean QEMU exited and the monitor won't work. It's better to simply terminate the process instead of trying to communicate using QMP. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 045dca5d02..00b44ea0f4 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -251,7 +251,7 @@ class QEMUMachine(object): self._launch() self._launched =3D True except: - self.shutdown() + self.shutdown(force=3DTrue) =20 LOG.debug('Error launching VM') if self._qemu_full_args: --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360516209805.202435509341; Thu, 29 Mar 2018 14:55:16 -0700 (PDT) Received: from localhost ([::1]:32771 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fVr-0007zw-Fq for importer@patchew.org; Thu, 29 Mar 2018 17:55:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33176) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHf-000318-6U for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHc-0002NB-3v for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52036) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHb-0002Ly-VJ for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:32 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 40E4F7F7B5; Thu, 29 Mar 2018 21:40:31 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 905EB600C9; Thu, 29 Mar 2018 21:40:30 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:54 -0300 Message-Id: <20180329213857.15499-16-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 29 Mar 2018 21:40:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 15/18] qemu.py: qmp_obj() method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" A wrapper for the QEMUMonitorProtocol.cmd_obj() method. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/qemu.py b/scripts/qemu.py index 00b44ea0f4..d25fe030bb 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -311,6 +311,9 @@ class QEMUMachine(object): =20 return self._qmp.cmd(cmd, args=3Dqmp_args) =20 + def qmp_obj(self, *args, **kwargs): + return self._qmp.cmd_obj(*args, **kwargs) + def command(self, cmd, conv_keys=3DTrue, **args): ''' Invoke a QMP command. --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360262927400.4461961066892; Thu, 29 Mar 2018 14:51:02 -0700 (PDT) Received: from localhost ([::1]:60243 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fRm-0003iG-5C for importer@patchew.org; Thu, 29 Mar 2018 17:51:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33210) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHk-00035x-Gg for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHj-0002e5-H1 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44500) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHj-0002c5-At for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:39 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96B5181DF5; Thu, 29 Mar 2018 21:40:38 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2BBF60BEC; Thu, 29 Mar 2018 21:40:32 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:55 -0300 Message-Id: <20180329213857.15499-17-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 29 Mar 2018 21:40:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 16/18] qemu.py: is_launched() method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Useful if some code gets a QEMUMachine object in unknown state, and wants to launch it only if necessary. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/qemu.py b/scripts/qemu.py index d25fe030bb..d32b923a15 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -236,6 +236,9 @@ class QEMUMachine(object): =20 self._launched =3D False =20 + def is_launched(self): + return self._launched + def launch(self): """ Launch the VM and make sure we cleanup and expose the --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360197474408.5461067254614; Thu, 29 Mar 2018 14:49:57 -0700 (PDT) Received: from localhost ([::1]:60235 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fQi-0002hO-Dl for importer@patchew.org; Thu, 29 Mar 2018 17:49:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33246) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fHr-0003JA-Od for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHp-0002ta-8A for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50388) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHo-0002rP-TR for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:45 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2EAEA12E373; Thu, 29 Mar 2018 21:40:44 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 075FF60BE0; Thu, 29 Mar 2018 21:40:42 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:56 -0300 Message-Id: <20180329213857.15499-18-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 29 Mar 2018 21:40:44 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 17/18] validator.py script X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" See cover letter for a description of the new test system. TODO: code clean up TODO: write description. Signed-off-by: Eduardo Habkost --- scripts/validator.py | 724 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 724 insertions(+) create mode 100755 scripts/validator.py diff --git a/scripts/validator.py b/scripts/validator.py new file mode 100755 index 0000000000..4312571feb --- /dev/null +++ b/scripts/validator.py @@ -0,0 +1,724 @@ +#!/usr/bin/env python +# +# Copyright (c) 2018 Red Hat Inc +# +# Author: +# Eduardo Habkost +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +""" +QEMU validator script +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This script will get test YAML test case specifications or Python +modules as input, and generate/run test cases based on them. + +USAGE +----- + +validator.py ... -V VAR1=3Dvalue1 VAR1=3Dvalue2 VAR2= =3Dvalue3 + +specification-file is a YAML file containing the test specification. + +Example:: + + # this test specification is equivalent to the + # "device/introspect/list" test case in device-introspect-test.c + command-line: '$QEMU -nodefaults -machine none' + monitor-commands: + - qmp: + - execute: qom-list-types + arguments: + implements: 'device' + abstract: true + - hmp: 'device_add help' + + +VARIABLE EXPANSION +------------------ + +The test runner will try to run the test cases with all possible values +for variables appearing in the test specification. + +Some built-in variables are automatically expanded: + +* `$MACHINE` - Expands to a machine-type name supported by $QEMU +* `$ACCEL` - Expands to an accelerator name supported by $QEMU +* `$DEVICE` - Expands to a (user-creatable) device type name supported by = $QEMU +* `$CPU` - Expands to a CPU model name supported by $QEMU + +Note that the $QEMU variable must be specified in th + +TEST SPECIFICATION FIELDS +------------------------- + +command-line +~~~~~~~~~~~~ + +List or string, containing the QEMU command-line to be run. + +Default: '$QEMU' + + +monitor-commands +~~~~~~~~~~~~~~~~ + +Mapping or list-of-mappings containing monitor commands to run. The key o= n each +item can be ``hmp`` or ``qmp``. The value on each entry can be a string, +mapping, or list. + +Default: None. + +TODO: not implemented yet. + + +qmp +~~~ + +Boolean. If true (the default), a QMP monitor is configured on the comman= d-line +automatically. + +If true, the test runner will issue a ``quit`` command automatically when +testing is finished. If false, the test runner will wait until QEMU exits= by +itself. + +Example:: + + # just run $QEMU -help and ensure it won't crash + command-line: ['$QEMU', '-help'] + qmp: false + + +TODO: whitelist +TODO: validate output against reference output +TODO: configure defaults for variables +TODO: compatibility with Avocado multiplexer? +""" + +import sys +import os +import string +import argparse +import pprint +import yaml +import logging +import shlex +import pipes +import re +import itertools +import traceback +import socket +from collections import OrderedDict + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'scripts')) +from qemu import QEMUMachine +from qmp.qmp import QMPError + +logger =3D logging.getLogger('qemu.tests.validator') +dbg =3D logger.debug + +# Python 2.7 compatibility: +shquote =3D getattr(shlex, 'quote', pipes.quote) + +class InvalidSpecification(Exception): + pass + +def qom_type_names(vm, **kwargs): + """Run qom-list-types QMP command, return type names""" + types =3D vm.command('qom-list-types', **kwargs) + return [t['name'] for t in types] + + +def info_qdm(vm): + """Parse 'info qdm' output""" + args =3D {'command-line': 'info qdm'} + devhelp =3D vm.command('human-monitor-command', **args) + for l in devhelp.split('\n'): + l =3D l.strip() + if l =3D=3D '' or l.endswith(':'): + continue + d =3D {'name': re.search(r'name "([^"]+)"', l).group(1), + 'no-user': (re.search(', no-user', l) is not None)} + yield d + + +class QemuBinaryInfo(object): + """Information for a specific QEMU binary""" + def __init__(self, binary): + """Don't instantiate this directly, use get_binary_info()""" + self.binary =3D binary + + args =3D ['-S', '-machine', 'none,accel=3Dkvm:tcg'] + dbg("querying info for QEMU binary: %s", binary) + vm =3D QEMUMachine(binary=3Dbinary, args=3Dargs) + vm.launch() + try: + self.alldevs =3D qom_type_names(vm, implements=3D'device', abs= tract=3DFalse) + # there's no way to query DeviceClass::user_creatable using QM= P, + # so use 'info qdm': + self.no_user_devs =3D [d['name'] for d in info_qdm(vm, ) if d[= 'no-user']] + self.machines =3D [m['name'] for m in vm.command('query-machin= es')] + self.user_devs =3D [dev for dev in self.alldevs if dev not in = self.no_user_devs] + self.kvm_available =3D vm.command('query-kvm')['enabled'] + self.cpu_models =3D [c['name'] for c in vm.command('query-cpu-= definitions')] + finally: + vm.shutdown() + + def available_accels(self): + if self.kvm_available: + yield 'kvm' + yield 'tcg' + +BINARY_INFO =3D {} + +def get_binary_info(binary): + """Lookup info for QEMU binary, caching data""" + if binary not in BINARY_INFO: + BINARY_INFO[binary] =3D QemuBinaryInfo(binary) + return BINARY_INFO[binary] + + +# TEST CASE GENERATION LOGIC +#TODO: the functions below are not used yet, refactor code to use +# the simpler variable expansion logic + +def updatedict(d1, d2): + """Return updated dictionary, after d1.update(d2)""" + d =3D d1.copy() + d.update(d2) + return d + +def newdict(d, k, v): + """Return new dictionary with d[k] =3D v + + >>> a =3D {'a':1, 'b':2} + >>> b =3D newdict(a, 'a', 100) + >>> sorted(a.items()) + [('a', 1), ('b', 2)] + >>> sorted(b.items()) + [('a', 100), ('b', 2)] + """ + d =3D d.copy() + d[k] =3D v + return d + +def mapchain(fn, l): + """map(fn, l) and then chain the results together + + >>> multiples =3D lambda i: [i, i*2, i*3] + >>> list(mapchain(multiples, [2, 3, 5])) + [2, 4, 6, 3, 6, 9, 5, 10, 15] + """ + return itertools.chain(*map(fn, l)) + +def call_func(v): + """Call function, if v is callable""" + if callable(v): + return v() + return v + +def fnitem(tc, var, fn): + """Replace tc[var] with fn(tc[var])""" + return newdict(tc, var, fn(tc[var])) + +def call_var_func(tc, var): + """Update `var` with result of its enumeration func, if necessary + + >>> retlist =3D lambda tc: [{'a':[1,2,3], 'c':['hi']}] + >>> tc =3D {'a':retlist, 'b':[100]} + >>> tc =3D call_var_func(tc, 'a') + >>> list(tc) =3D=3D [{'a':[1,2,3], 'b':[100], 'c':['hi']}] + True + """ + if callable(tc[var]): + r =3D tc[var](tc) + assert all(var in t for t in r) + return (updatedict(tc, t) for t in r) + else: + return [tc] + +#def simple_enum(fn): +# """Wrapper for simple enumeration functions that return a list of val= ues +# +# >>> fn =3D simple_enum(lambda: [1,2,3]) +# >>> tc =3D {'a':fn} +# >>> tc =3D call_var_func(tc, 'a') +# >>> tc['a'] +# [1, 2, 3] +# """ +# def enum_func(tc): +# return {} + +def split_var(tc, var): + """Split test case into multiple items + + >>> tc =3D {'a':[1,2,3], 'b':100} + >>> tc =3D split_var(tc, 'a') + >>> [r['a'] for r in tc] + [[1], [2], [3]] + """ + return (newdict(tc, var, [v]) for v in tc[var]) + +def split_vars(tc, vars): + """Split test case for multiple variables + + >>> list1 =3D [1,2] + >>> list2 =3D [10, 20] + >>> tc =3D {'a':list1, 'b':list2, 'c':[100, 200, 300]} + >>> tc =3D split_vars(tc, ['a', 'b']) + >>> list(tc) =3D=3D [{'a':[i], 'b':[j], 'c':[100, 200, 300]} for i in = [1,2] for j in [10, 20]] + True + """ + tcs =3D [tc] + for var in vars: + #TODO: call_var_func() + tcs =3D mapchain(lambda tc: split_var(tc, var), tcs) + return tcs + +class VariableDefinition(object): + # variable names that must be set before this variable + deps =3D [] + + def __init__(self): + self.values =3D None + + @staticmethod + def enumerate(values): + """Enumerate possible values for the variable""" + raise NotImplementedError + +class BuiltinVars(object): + """Namespace for builtin variables""" + class QEMU(VariableDefinition): + """QEMU binary path, must be set explicitly by the user""" + def enumerate(self, values): + raise Exception("QEMU variable not set") + + class MACHINE(VariableDefinition): + """Machine-type name. Don't use with $MACHINE_OPT""" + deps =3D ['QEMU'] + + @staticmethod + def enumerate(values): + return get_binary_info(values['QEMU']).machines + + class ACCEL(VariableDefinition): + deps =3D ['QEMU'] + + @staticmethod + def enumerate(values): + return get_binary_info(values['QEMU']).available_accels() + + class DEVICE(VariableDefinition): + deps =3D ['QEMU'] + + @staticmethod + def enumerate(values): + return get_binary_info(values['QEMU']).user_devs + + class CPU(VariableDefinition): + deps =3D ['QEMU'] + + @staticmethod + def enumerate(values): + return get_binary_info(values['QEMU']).cpu_models + +class VariableEnumerator(object): + """Helper class that will enumerate possible values for variables""" + def __init__(self): + # start with built-in variables: + self._vars =3D dict((v, getattr(BuiltinVars, v)()) for v in dir(Bu= iltinVars) if not v.startswith('_')) + self._defaults =3D {} + + def _add_var(self, var): + return self._vars.setdefault(var, VariableDefinition()) + + def set_values(self, var, values): + """Set default values for variable `var` + + `values` can be a list of values, or a single item + Default values override the values returned by VariableDefinition.= enumerate() + """ + if not isinstance(values, list): + values =3D [values] + self._add_var(var).values =3D values + + def update_values(self, valuedict): + dbg("update_values: %r", valuedict) + for k,v in valuedict.items(): + self.set_values(k, v) + + def lookup_var(self, var): + """Lookup variable""" + return self._vars.get(var) + + def var_deps(self, var): + """Return variables need to be set before `var`""" + return self.lookup_var(var).deps + + def order_deps(self, vars): + """Return full list of variables, including dependencies in the ri= ght order + """ + dbg("START order_deps %r", vars) + result =3D OrderedDict() + vars =3D OrderedDict(((v,1) for v in reversed(vars))) + while vars: + dbg("queue: %r", list(vars.keys())) + v,_ =3D vars.popitem(last=3DTrue) + dbg("next var: %s", v) + deps =3D self.var_deps(v) + dbg("deps: %r", deps) + if all(dep in result for dep in deps): + dbg("var %s is ready", v) + result[v] =3D 1 + continue + # dependencies not met: + vars[v] =3D 1 + for dep in deps: + if dep in vars: + raise Exception("Variable dependency cycle: %s" % (' -= > '.join(vars.keys()))) + vars[dep] =3D 1 + return list(result.keys()) + + def enumerate(self, env, var, values): + """Enumerate possible values for variable `var` + + May be called only if all requirements for `var` are set in `value= s`. + """ + v =3D self.lookup_var(var) + assert all(dep in values for dep in v.deps) + dbg("var: %r", v) + if v.values: + r =3D v.values + else: + r =3D v.enumerate(values) + logger.debug('Values for %s: %r', var, r) + return r + +# HELPER FUNCTIONS FOR TEMPLATE STRINGS: + +def apply_template(templ, values): + """Apply variables to a template, supporting strings and lists + + >>> apply_template('$QEMU -machine X', {'QEMU':'qemu-system-x86_64'}) + 'qemu-system-x86_64 -machine X' + >>> apply_template({"$TEST": ["$FOO", "is $BAR"]}, \ + {'FOO':'XX', 'BAR':'YY', 'TEST':'TT'}) + {'$TEST': ['XX', 'is YY']} + """ + if isinstance(templ, str): + return string.Template(templ).substitute(values) + elif isinstance(templ, list): + return [apply_template(s, values) for s in templ] + elif isinstance(templ, dict): + return dict( (k, apply_template(v, values)) for (k, v) in templ.it= ems()) + else: + return templ + +def vars_for_template(templ): + """Return list of variables used by s when used as template string + + >>> vars_for_template('abcde fgh') + [] + >>> vars_for_template('$A is ${A}, not ${B} or $C') + ['A', 'B', 'C'] + >>> vars_for_template(['$QEMU', '-machine' , '$MACHINE$MACHINE_OPT']) + ['QEMU', 'MACHINE', 'MACHINE_OPT'] + """ + usedKeys =3D OrderedDict() + class LoggingDict(object): + def __getitem__(self, k): + usedKeys[k] =3D 1 + return 'X' + apply_template(templ, LoggingDict()) + return list(usedKeys.keys()) + +class TestSpecification(object): + def __init__(self, data): + self._data =3D data + self.normalize() + + def normalize(self): + """Normalize test specification data + + * ensure 'command-line' is a list of arguments + * 'monitor-commands' will be an array + """ + # if command-line is omitted, just run QEMU with no arguments: + self._data.setdefault('command-line', ['$QEMU']) + self._data.setdefault('monitor-commands', []) + + # 'monitor-commands' must be a list + if not isinstance(self.get('monitor-commands'), list): + self._data['monitor-commands'] =3D [self.get('monitor-commands= ')] + + @classmethod + def load_file(cls, file): + data =3D yaml.load(open(file)) + return cls(data) + + def get(self, key, default=3DNone): + return self._data.get(key, default) + + def _gen_subtests(self, testcases, env, vars): + """Call _gen_test_cases() for each test case in `testcases`""" + #import pdb; pdb.set_trace() + for tc in testcases: + for st in self._gen_test_cases(env, vars, tc): + yield st + + def _gen_var_testcases(self, env, var, values): + """Generate one test case for each possible value for `var`""" + for value in self.var_enum.enumerate(env, var, values): + newvalues =3D values.copy() + newvalues[var] =3D value + yield newvalues + + def _gen_test_cases(self, env, vars, values): + """Generate list of test cases + + :param vars: List of variable names that are not set yet + :param values: values of variables that are already set + """ + if not vars: + # No unset variables -> only 1 test case + return [values.copy()] + + # pick next unset variable, enumerate values, set it: + var =3D vars[0] + cases =3D self._gen_var_testcases(env, var, values) + cases =3D self._gen_subtests(cases, env, vars[1:]) + return cases + + def gen_test_cases(self, env): + """Generate all test cases for this test specification""" + # we generate combinations for the command-line and monitor comman= ds + # in separate steps, so test cases using the same QEMU command-line + # are grouped together + vars =3D vars_for_template(self.get('command-line')) + vars_for_te= mplate(self.get('monitor-commands')) + + self.var_enum =3D VariableEnumerator() + if not env.args.full: + self.var_enum.update_values(self.get('defaults', {})) + self.var_enum.update_values(env.var_values) + + # put dependencies in right order: + vars =3D self.var_enum.order_deps(vars) + cases =3D self._gen_test_cases(env, vars, {}) + return (TestCase(self, c) for c in cases) + +class TestCase(object): + def __init__(self, spec, values): + self.spec =3D spec + self.values =3D values + + def __str__(self): + return ' '.join('%s=3D%s' % (k, shquote(v)) for k,v in self.values= .items()) + + def is_expected_entry(self, expected_entry): + """Check if `expected_entry` matches the testcase/results""" + expected_vars =3D expected_entry.copy() + for var,value in expected_vars.items(): + if self.values.get(var) !=3D value: + return False + return True + + def is_expected_failure(self): + for e in self.getField('expected-failures', []): + if self.is_expected_entry(e): + return True + + def getField(self, var, default=3DNone): + """Get value of test spec field, expanding variables""" + return apply_template(self.spec.get(var, default), self.values) + + def qmp_cmd(self, vm, cmd): + if isinstance(cmd, list): + for c in cmd: + self.qmp_cmd(vm, c) + elif isinstance(cmd, dict): + return vm.qmp_obj(cmd) + else: + raise InvalidSpecification("QMP command must be dict: %r" % (c= md)) + + def hmp_cmd(self, vm, cmd): + return vm.command('human-monitor-command', command_line=3Dcmd) + + def monitor_cmd(self, vm, cmd): + dbg("monitor cmd: %r", cmd) + if isinstance(cmd, dict): + for k,v in cmd.items(): + if k =3D=3D 'qmp': + self.qmp_cmd(vm, v) + elif k =3D=3D 'hmp': + self.hmp_cmd(vm, v) + else: + raise InvalidSpecification("Invalid monitor command: %= r: %r" % (k, v)) + + def run(self, env): + """Check one specific test case + + Returns a dictionary containing failure information on error, + or None on success + """ + result =3D {'success': True } + result['is-expected-failure'] =3D self.is_expected_failure() + + cmdline =3D self.getField('command-line') + if not isinstance(cmdline, list): + cmdline =3D shlex.split(cmdline) + + qmp =3D self.getField('qmp', True) + #TODO: use context manager to enter/exit borrowed VM from env + vm =3D env.get_vm(cmdline, qmp) + try: + if not vm.is_launched(): + vm.launch() + #TODO: generate/enumerate variables inside monitor commands too + for cmd in self.getField('monitor-commands', []): + self.monitor_cmd(vm, cmd) + if not qmp: + vm.wait() + env.drop_vm() + except KeyboardInterrupt: + raise + except QMPError as err: + result['exception'] =3D repr(err) + result['success'] =3D False + except socket.error as err: + result['exception'] =3D repr(err) + result['success'] =3D False + + dbg('vm is %r', vm) + ec =3D vm.exitcode() + dbg("exit code: %r", ec) + if ec is not None and ec !=3D 0: + result['success'] =3D False + result['exitcode'] =3D ec + result['log'] =3D vm.get_log() + + #TODO: use context manager to enter/exit borrowed VM from env + if not result['success']: + env.drop_vm() + + return result + + +class TestEnv(object): + def __init__(self, args): + self.args =3D args + self._last_vm_args =3D None + self._last_vm =3D None + + def qemu_binaries(self): + return self.args.qemu_binaries + + def drop_vm(self): + """Drop existing VM object""" + if self._last_vm: + #TODO: record failures here + self._last_vm.shutdown() + self._last_vm =3D None + self._last_vm_args =3D None + + def get_vm(self, cmdline, qmp): + """Get VM object for test case""" + if self._last_vm_args =3D=3D (cmdline, qmp) and self._last_vm.is_r= unning(): + dbg("Reusing VM object for cmdline %r", cmdline) + return self._last_vm + + dbg("Starting new VM for cmdline %r", cmdline) + #FIXME: need to catch exitcode/segfaults here somehow :( + self.drop_vm() + vm =3D QEMUMachine(binary=3Dcmdline[0], args=3Dcmdline[1:], qmp=3D= qmp) + self._last_vm =3D vm + self._last_vm_args =3D (cmdline, qmp) + return vm + +def main(): + parser =3D argparse.ArgumentParser(description=3D"Generic QEMU validat= or") + parser.set_defaults(loglevel=3Dlogging.INFO) + parser.add_argument('-V', metavar=3D'VAR=3DVALUE', nargs=3D'*', + help=3D"Force variabie VAR to VALUE", + action=3D'append', dest=3D'vars', default=3D[]) + parser.add_argument('-d', '--debug',action=3D'store_const', + dest=3D'loglevel', const=3Dlogging.DEBUG, + help=3D'debug output') + parser.add_argument('-v', '--verbose',action=3D'store_const', + dest=3D'loglevel', const=3Dlogging.INFO, + help=3D'verbose output') + parser.add_argument('-q', '--quiet',action=3D'store_const', + dest=3D'loglevel', const=3Dlogging.WARN, + help=3D'non-verbose output') + parser.add_argument("--dry-run", action=3D"store_true", + help=3D"Don't run test cases") + parser.add_argument("--full", action=3D"store_true", + help=3D"Run all test case combinations, not just t= he default for the test specification") + parser.add_argument("testfiles", nargs=3D"+", metavar=3D"FILE", + help=3D"Load test case specification from FILE") + args =3D parser.parse_args() + + env =3D TestEnv(args) + + vars =3D {} + if args.vars: + for varval in itertools.chain(*args.vars): + var,val =3D varval.split('=3D', 1) + vars.setdefault(var, []).append(val) + env.var_values =3D vars + + logging.basicConfig(stream=3Dsys.stdout, level=3Dargs.loglevel, format= =3D'%(levelname)s: %(message)s') + resultdict =3D {} + try: + for testfile in args.testfiles: + specname =3D os.path.basename(testfile) + #TODO: support test specifications pointing to Python modules + spec =3D TestSpecification.load_file(testfile) + logger.debug("Test specification:") + logger.debug(pprint.pformat(spec._data)) + logger.debug('---') + for tc in spec.gen_test_cases(env): + if tc.is_expected_failure(): + logger.info("%s: Skipped: %s", specname, str(tc)) + continue + logger.info("%s: Running: %s", specname, str(tc)) + if not args.dry_run: + r =3D tc.run(env) + logger.debug("Result:") + logger.debug(pprint.pformat(r)) + if not r['success']: + logger.error("%s: failed: %s", specname, tc) + resultdict.setdefault(r['success'], []).append( (tc, r= ) ) + except KeyboardInterrupt: + # Print partial test result summary on interrupt + logger.info("Interrupted. Partial test summary follows") + pass + + env.drop_vm() + + if not args.dry_run: + logger.info('%d successes', len(resultdict.get(True, []))) + failures =3D resultdict.get(False, []) + if failures: + logger.error('%d failures', len(failures)) + for tc,r in failures: + logger.error("Failed: %s", tc) + logger.error("Result:") + pprint.pprint(r) + dbg("Result: %r", r) + +if __name__ =3D=3D '__main__': + sys.exit(main()) --=20 2.14.3 From nobody Thu May 2 15:49:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522360595712234.0234086078725; Thu, 29 Mar 2018 14:56:35 -0700 (PDT) Received: from localhost ([::1]:32855 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fX8-0000TJ-Va for importer@patchew.org; Thu, 29 Mar 2018 17:56:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33304) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1fI0-0003Ud-6e for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1fHx-00035m-30 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58298) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1fHw-00034j-Qs for qemu-devel@nongnu.org; Thu, 29 Mar 2018 17:40:53 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1B6DB28209; Thu, 29 Mar 2018 21:40:52 +0000 (UTC) Received: from localhost (ovpn-116-15.gru2.redhat.com [10.97.116.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89F656A96A; Thu, 29 Mar 2018 21:40:45 +0000 (UTC) From: Eduardo Habkost To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 18:38:57 -0300 Message-Id: <20180329213857.15499-19-ehabkost@redhat.com> In-Reply-To: <20180329213857.15499-1-ehabkost@redhat.com> References: <20180329213857.15499-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 29 Mar 2018 21:40:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 18/18] Collection of validator.py test cases X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Amador Pahim , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Cleber Rosa , Marcel Apfelbaum , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" These test case specifications demonstrate what kind of tests can be specified using validator.py. Most of them can replace existing unit tests written in C, and device-crash-test.yaml can replace most of the functionality of scripts/device-crash-test. Signed-off-by: Eduardo Habkost --- tests/validator/cpu.yaml | 5 +++++ tests/validator/device-add.yaml | 17 +++++++++++++++++ tests/validator/device-crash-test.yaml | 7 +++++++ tests/validator/device-help.yaml | 3 +++ tests/validator/device-introspect.yaml | 23 +++++++++++++++++++++++ tests/validator/device-list.yaml | 10 ++++++++++ tests/validator/just-help.yaml | 3 +++ 7 files changed, 68 insertions(+) create mode 100644 tests/validator/cpu.yaml create mode 100644 tests/validator/device-add.yaml create mode 100644 tests/validator/device-crash-test.yaml create mode 100644 tests/validator/device-help.yaml create mode 100644 tests/validator/device-introspect.yaml create mode 100644 tests/validator/device-list.yaml create mode 100644 tests/validator/just-help.yaml diff --git a/tests/validator/cpu.yaml b/tests/validator/cpu.yaml new file mode 100644 index 0000000000..48032995ff --- /dev/null +++ b/tests/validator/cpu.yaml @@ -0,0 +1,5 @@ +# Simply check if QEMU will run without errors using a CPU model: +command-line: '$QEMU -S -cpu $CPU' +expected-failures: + # "host" doesn't always work + - CPU: 'host' diff --git a/tests/validator/device-add.yaml b/tests/validator/device-add.y= aml new file mode 100644 index 0000000000..b4c0daf70c --- /dev/null +++ b/tests/validator/device-add.yaml @@ -0,0 +1,17 @@ +# This test specification will try device_add with all device types +command-line: '$QEMU -S -machine $MACHINE,accel=3D$ACCEL' +monitor-commands: +- qmp: + - execute: 'device_add' + arguments: + driver: '$DEVICE' + id: 'test-device-for-$DEVICE' + - execute: 'device_del' + arguments: + id: 'test-device-for-$DEVICE' +defaults: + MACHINE: 'none' + ACCEL: 'kvm:tcg' +expected-failures: + - MACHINE: 'xenpv' + - MACHINE: 'xenfv' diff --git a/tests/validator/device-crash-test.yaml b/tests/validator/devic= e-crash-test.yaml new file mode 100644 index 0000000000..edaab0ee8f --- /dev/null +++ b/tests/validator/device-crash-test.yaml @@ -0,0 +1,7 @@ +# this is a very simple test case generator that will run QEMU +# using all combinations of -machine and -device. +# This replaces scripts/device-crash-test. +# TODO: use a $MACHINE_OPT variable to make -machine optional +command-line: '$QEMU -S -machine $MACHINE,accel=3D$ACCEL -device $DEVICE' + +#TODO: represent the whitelist from device-crash-test script somehow diff --git a/tests/validator/device-help.yaml b/tests/validator/device-help= .yaml new file mode 100644 index 0000000000..95aa5c9f5b --- /dev/null +++ b/tests/validator/device-help.yaml @@ -0,0 +1,3 @@ +# Just check if -device ...,help works: +command-line: '$QEMU -device $DEVICE,help' +qmp: false diff --git a/tests/validator/device-introspect.yaml b/tests/validator/devic= e-introspect.yaml new file mode 100644 index 0000000000..f5b10aff9a --- /dev/null +++ b/tests/validator/device-introspect.yaml @@ -0,0 +1,23 @@ +# This test specification is equivalent to "device/introspect/concrete" in +# tests/device-introspect-test.c +command-line: '$QEMU -S -machine $MACHINE,accel=3D$ACCEL' +monitor-commands: +- qmp: + - execute: 'device-list-properties' + arguments: + typename: '$DEVICE' +- hmp: 'device_add $DEVICE,help' +- hmp: 'info qom-tree' +defaults: + MACHINE: 'none' + ACCEL: 'kvm:tcg' +expected-failures: + - MACHINE: 'xenpv' + - MACHINE: 'xenfv' + +#TODO: the test runner could support something like: +# extra-cases: +# # like "device/introspect/none": +# - DEVICE: 'nonexistent' +# # like "device/introspect/abstract": +# - DEVICE: 'device' diff --git a/tests/validator/device-list.yaml b/tests/validator/device-list= .yaml new file mode 100644 index 0000000000..ac0004c05d --- /dev/null +++ b/tests/validator/device-list.yaml @@ -0,0 +1,10 @@ +# this test specification is equivalent to the +# "device/introspect/list" test case in device-introspect-test.c +command-line: '$QEMU -nodefaults -machine none' +monitor-commands: +- qmp: + - execute: qom-list-types + arguments: + implements: 'device' + abstract: true +- hmp: 'device_add help' diff --git a/tests/validator/just-help.yaml b/tests/validator/just-help.yaml new file mode 100644 index 0000000000..84ec8d7090 --- /dev/null +++ b/tests/validator/just-help.yaml @@ -0,0 +1,3 @@ +# just run $QEMU -help and ensure it won't crash +command-line: '$QEMU -help' +qmp: false --=20 2.14.3