From nobody Sat Apr 27 14:17:25 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1490199189187457.20682737030734; Wed, 22 Mar 2017 09:13:09 -0700 (PDT) Received: from localhost ([::1]:52002 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqisl-00048X-IX for importer@patchew.org; Wed, 22 Mar 2017 12:13:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34876) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqihN-0002d0-Gl for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cqihH-0006t0-S8 for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:21 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42248) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cqihH-0006sX-Le for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:15 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9A14361D01 for ; Wed, 22 Mar 2017 16:01:15 +0000 (UTC) Received: from localhost (ovpn-116-17.gru2.redhat.com [10.97.116.17]) by smtp.corp.redhat.com (Postfix) with ESMTP id 484A478F04; Wed, 22 Mar 2017 16:01:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9A14361D01 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 9A14361D01 From: Eduardo Habkost To: qemu-devel@nongnu.org, Markus Armbruster Date: Wed, 22 Mar 2017 13:00:50 -0300 Message-Id: <20170322160052.2820-2-ehabkost@redhat.com> In-Reply-To: <20170322160052.2820-1-ehabkost@redhat.com> References: <20170322160052.2820-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 22 Mar 2017 16:01:15 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 1/3] qemu.py: Always save QEMU exit code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marcel Apfelbaum , Thomas Huth Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Instead of saving the QEMU exit code only if shutdown() gets called, move it to _post_shutdown(), so it gets saved even if the launch() method has failed. Add a exitcode() method that scripts can use to query the QEMU exit code. Signed-off-by: Eduardo Habkost --- scripts/qemu.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 6d1b6230b7..c9f35b97f5 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -39,6 +39,7 @@ class QEMUMachine(object): self._iolog =3D None self._socket_scm_helper =3D socket_scm_helper self._debug =3D debug + self._exitcode =3D None =20 # This can be used to add an unused monitor instance. def add_monitor_telnet(self, ip, port): @@ -116,6 +117,9 @@ class QEMUMachine(object): if not isinstance(self._monitor_address, tuple): self._remove_if_exists(self._monitor_address) self._remove_if_exists(self._qemu_log_path) + self._exitcode =3D self._popen.wait() + if self._exitcode < 0: + sys.stderr.write('qemu received signal %i: %s\n' % (-self._exi= tcode, ' '.join(self._args))) =20 def launch(self): '''Launch the VM and establish a QMP connection''' @@ -135,6 +139,9 @@ class QEMUMachine(object): self._popen =3D None raise =20 + def exitcode(self): + return self._exitcode + def shutdown(self): '''Terminate the VM and clean up''' if not self._popen is None: @@ -144,9 +151,6 @@ class QEMUMachine(object): except: self._popen.kill() =20 - exitcode =3D self._popen.wait() - if exitcode < 0: - sys.stderr.write('qemu received signal %i: %s\n' % (-exitc= ode, ' '.join(self._args))) self._load_io_log() self._post_shutdown() self._popen =3D None --=20 2.11.0.259.g40922b1 From nobody Sat Apr 27 14:17:25 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 14901986604401003.4848330245478; Wed, 22 Mar 2017 09:04:20 -0700 (PDT) Received: from localhost ([::1]:51899 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqikE-0004sn-ET for importer@patchew.org; Wed, 22 Mar 2017 12:04:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34907) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqihP-0002eu-H8 for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cqihK-0006u3-10 for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60338) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cqihJ-0006tf-Rg for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:17 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C76AF6E77A for ; Wed, 22 Mar 2017 16:01:17 +0000 (UTC) Received: from localhost (ovpn-116-17.gru2.redhat.com [10.97.116.17]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0A72D7F381; Wed, 22 Mar 2017 16:01:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C76AF6E77A Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C76AF6E77A From: Eduardo Habkost To: qemu-devel@nongnu.org, Markus Armbruster Date: Wed, 22 Mar 2017 13:00:51 -0300 Message-Id: <20170322160052.2820-3-ehabkost@redhat.com> In-Reply-To: <20170322160052.2820-1-ehabkost@redhat.com> References: <20170322160052.2820-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 22 Mar 2017 16:01:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 2/3] qtest.py: Support QTEST_LOG environment variable X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marcel Apfelbaum , Thomas Huth Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" qtest logs everything to stderr by default, but we don't want it to be the default behavior on test cases. Implement the same behavior of libqtest.c, and redirect the qtest log to /dev/null by default unless the QTEST_LOG environment variable is set. Signed-off-by: Eduardo Habkost --- Patch originally submitted as part of series: * [RFC v2 00/20] qmp: Report bus information on 'query-machines' --- scripts/qtest.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/qtest.py b/scripts/qtest.py index d5aecb5f49..5ac2c69ba8 100644 --- a/scripts/qtest.py +++ b/scripts/qtest.py @@ -88,8 +88,14 @@ class QEMUQtestMachine(qemu.QEMUMachine): self._qtest_path =3D os.path.join(test_dir, name + "-qtest.sock") =20 def _base_args(self): + if os.getenv('QTEST_LOG'): + qtest_log =3D '/dev/fd/2' + else: + qtest_log =3D '/dev/null' + args =3D super(QEMUQtestMachine, self)._base_args() args.extend(['-qtest', 'unix:path=3D' + self._qtest_path, + '-qtest-log', qtest_log, '-machine', 'accel=3Dqtest']) return args =20 --=20 2.11.0.259.g40922b1 From nobody Sat Apr 27 14:17:25 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1490199026878463.02699030870235; Wed, 22 Mar 2017 09:10:26 -0700 (PDT) Received: from localhost ([::1]:51981 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqiq9-0001n7-Dd for importer@patchew.org; Wed, 22 Mar 2017 12:10:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34960) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqihT-0002jH-Sw for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cqihQ-0006xk-SA for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:21825) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cqihQ-0006xD-GE for qemu-devel@nongnu.org; Wed, 22 Mar 2017 12:01:24 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6F6177EBA3 for ; Wed, 22 Mar 2017 16:01:24 +0000 (UTC) Received: from localhost (ovpn-116-17.gru2.redhat.com [10.97.116.17]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30D857E626; Wed, 22 Mar 2017 16:01:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6F6177EBA3 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 6F6177EBA3 From: Eduardo Habkost To: qemu-devel@nongnu.org, Markus Armbruster Date: Wed, 22 Mar 2017 13:00:52 -0300 Message-Id: <20170322160052.2820-4-ehabkost@redhat.com> In-Reply-To: <20170322160052.2820-1-ehabkost@redhat.com> References: <20170322160052.2820-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 22 Mar 2017 16:01:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 3/3] scripts: Test script to look for -device crashes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marcel Apfelbaum , Thomas Huth Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Test code to check if we can crash QEMU using -device. It will test all accel/machine/device combinations by default, which may take a few hours (it's more than 90k test cases). There's a "-r" option that makes it test a random sample of combinations. The scripts contains a whitelist for: 1) known error messages that make QEMU exit cleanly; 2) known QEMU crashes. This is the behavior when the script finds a failure: * Known clean (exitcode=3D1) error messages generate INFO messages (visible only in verbose mode), to make script output shorter * Unknown clean error messages generate warnings (visible by default) * Known crashes generate error messages, but are not fatal * Unknown crashes generate fatal error messages I'm unsure about the need to maintain a list of known clean error messages, but I wanted to at least document all existing failure cases to use as base to build more comprehensive test code. Signed-off-by: Eduardo Habkost --- scripts/device-crash-test.py | 486 +++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 486 insertions(+) create mode 100755 scripts/device-crash-test.py diff --git a/scripts/device-crash-test.py b/scripts/device-crash-test.py new file mode 100755 index 0000000000..693ed5c11e --- /dev/null +++ b/scripts/device-crash-test.py @@ -0,0 +1,486 @@ +#!/usr/bin/env python2.7 +# +# Run QEMU with all combinations of -machine and -device types, +# check for crashes and unexpected errors. +# +# Copyright (c) 2017 Red Hat Inc +# +# Author: +# Eduardo Habkost +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, see . +# + +import sys, os, glob +sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'scripts')) + +from qtest import QEMUQtestMachine +import logging, traceback, re, random, argparse + +logger =3D logging.getLogger('device-crash-test') +dbg =3D logger.debug + +# Valid whitelist entry keys: +# - accel: regexp, full match only +# - machine: regexp, full match only +# - device: regexp, full match only +# - log: regexp, partial match allowed +# - exitcode: if not present, defaults to 1. If None, matches any exitcode +# - warn: if True, matching failures will be logged as warnings +ERROR_WHITELIST =3D [ + # Machines that won't work out of the box: + # MACHINE | ERROR MESSAGE + dict(machine=3D'niagara'), # Unable to load a firmware for -M niag= ara + dict(machine=3D'boston'), # Please provide either a -kernel or -b= ios argument + dict(machine=3D'leon3_generic'), # Can't read bios image (null) + + # devices that don't work out of the box because they require extra opti= ons to "-device DEV": + # DEVICE | ERROR MESSAGE + dict(device=3D'.*-(i386|x86_64)-cpu'), # CPU socket-id is not set + dict(device=3D'ARM,bitband-memory'), # source-memory property not s= et + dict(device=3D'arm.cortex-a9-global-timer'), # a9_gtimer_realize: num-cp= u must be between 1 and 4 + dict(device=3D'arm_mptimer'), # num-cpu must be between 1 an= d 4 + dict(device=3D'armv7m'), # memory property was not set + dict(device=3D'aspeed.scu'), # Unknown silicon revision: 0x0 + dict(device=3D'aspeed.sdmc'), # Unknown silicon revision: 0x0 + dict(device=3D'bcm2835-dma'), # bcm2835_dma_realize: require= d dma-mr link not found: Property '.dma-mr' not found + dict(device=3D'bcm2835-fb'), # bcm2835_fb_realize: required= vcram-base property not set + dict(device=3D'bcm2835-mbox'), # bcm2835_mbox_realize: requir= ed mbox-mr link not found: Property '.mbox-mr' not found + dict(device=3D'bcm2835-peripherals'), # bcm2835_peripherals_realize:= required ram link not found: Property '.ram' not found + dict(device=3D'bcm2835-property'), # bcm2835_property_realize: re= quired fb link not found: Property '.fb' not found + dict(device=3D'bcm2835_gpio'), # bcm2835_gpio_realize: requir= ed sdhci link not found: Property '.sdbus-sdhci' not found + dict(device=3D'bcm2836'), # bcm2836_realize: required ra= m link not found: Property '.ram' not found + dict(device=3D'cfi.pflash01'), # attribute "sector-length" no= t specified or zero. + dict(device=3D'cfi.pflash02'), # attribute "sector-length" no= t specified or zero. + dict(device=3D'icp'), # icp_realize: required link '= xics' not found: Property '.xics' not found + dict(device=3D'ics'), # ics_base_realize: required l= ink 'xics' not found: Property '.xics' not found + dict(device=3D'ide-drive'), # No drive specified + dict(device=3D'ide-hd'), # No drive specified + dict(device=3D'ipmi-bmc-extern'), # IPMI external bmc requires c= hardev attribute + dict(device=3D'isa-debugcon'), # Can't create serial device, = empty char device + dict(device=3D'isa-ipmi-bt'), # IPMI device requires a bmc a= ttribute to be set + dict(device=3D'isa-ipmi-kcs'), # IPMI device requires a bmc a= ttribute to be set + dict(device=3D'isa-parallel'), # Can't create serial device, = empty char device + dict(device=3D'isa-serial'), # Can't create serial device, = empty char device + dict(device=3D'ivshmem'), # You must specify either 'shm= ' or 'chardev' + dict(device=3D'ivshmem-doorbell'), # You must specify a 'chardev' + dict(device=3D'ivshmem-plain'), # You must specify a 'memdev' + dict(device=3D'kvm-pci-assign'), # no host device specified + dict(device=3D'loader'), # please include valid argumen= ts + dict(device=3D'nand'), #' Unsupported NAND block size= 0x1 + dict(device=3D'nvdimm'), # 'memdev' property is not set + dict(device=3D'nvme'), # Device initialization failed + dict(device=3D'pc-dimm'), # 'memdev' property is not set + dict(device=3D'pci-bridge'), # Bridge chassis not specified= . Each bridge is required to be assigned a unique chassis id > 0. + dict(device=3D'pci-bridge-seat'), # Bridge chassis not specified= . Each bridge is required to be assigned a unique chassis id > 0. + dict(device=3D'pci-serial'), # Can't create serial device, = empty char device + dict(device=3D'pci-serial-2x'), # Can't create serial device, = empty char device + dict(device=3D'pci-serial-4x'), # Can't create serial device, = empty char device + dict(device=3D'pxa2xx-dma'), # channels value invalid + dict(device=3D'pxb'), # Bridge chassis not specified= . Each bridge is required to be assigned a unique chassis id > 0. + dict(device=3D'scsi-block'), # drive property not set + dict(device=3D'scsi-disk'), # drive property not set + dict(device=3D'scsi-generic'), # drive property not set + dict(device=3D'scsi-hd'), # drive property not set + dict(device=3D'spapr-pci-host-bridge'), # BUID not specified for PHB + dict(device=3D'spapr-pci-vfio-host-bridge'), # BUID not specified for PHB + dict(device=3D'spapr-rng'), #' spapr-rng needs an RNG back= end! + dict(device=3D'spapr-vty'), # chardev property not set + dict(device=3D'tpm-tis'), # tpm_tis: backend driver with= id (null) could not be found + dict(device=3D'unimplemented-device'), # property 'size' not specifie= d or zero + dict(device=3D'usb-braille'), # Property chardev is required + dict(device=3D'usb-mtp'), # x-root property must be conf= igured + dict(device=3D'usb-redir'), # Parameter 'chardev' is missi= ng + dict(device=3D'usb-serial'), # Property chardev is required + dict(device=3D'usb-storage'), # drive property not set + dict(device=3D'vfio-amd-xgbe'), # -device vfio-amd-xgbe: vfio = error: wrong host device name + dict(device=3D'vfio-calxeda-xgmac'), # -device vfio-calxeda-xgmac: = vfio error: wrong host device name + dict(device=3D'vfio-pci'), # No provided host device + dict(device=3D'vfio-pci-igd-lpc-bridge'), # VFIO dummy ISA/LPC bridge mu= st have address 1f.0 + dict(device=3D'vhost-scsi.*'), # vhost-scsi: missing wwpn + dict(device=3D'vhost-vsock-device'), # guest-cid property must be g= reater than 2 + dict(device=3D'vhost-vsock-pci'), # guest-cid property must be g= reater than 2 + dict(device=3D'virtio-9p-ccw'), # 9pfs device couldn't find fs= dev with the id =3D NULL + dict(device=3D'virtio-9p-device'), # 9pfs device couldn't find fs= dev with the id =3D NULL + dict(device=3D'virtio-9p-pci'), # 9pfs device couldn't find fs= dev with the id =3D NULL + dict(device=3D'virtio-blk-ccw'), # drive property not set + dict(device=3D'virtio-blk-device'), # drive property not set + dict(device=3D'virtio-blk-device'), # drive property not set + dict(device=3D'virtio-blk-pci'), # drive property not set + dict(device=3D'virtio-crypto-ccw'), # 'cryptodev' parameter expect= s a valid object + dict(device=3D'virtio-crypto-device'), # 'cryptodev' parameter expect= s a valid object + dict(device=3D'virtio-crypto-pci'), # 'cryptodev' parameter expect= s a valid object + dict(device=3D'virtio-input-host-device'), # evdev property is required + dict(device=3D'virtio-input-host-pci'), # evdev property is required + dict(device=3D'xen-pvdevice'), # Device ID invalid, it must a= lways be supplied + dict(device=3D'vhost-vsock-ccw'), # guest-cid property must be g= reater than 2 + dict(device=3D'ALTR.timer'), # "clock-frequency" property m= ust be provided + dict(device=3D'zpci'), # target must be defined + + # ioapic devices are already created by pc and will fail: + dict(machine=3D'pc-.*', device=3D'kvm-ioapic'), # Only 1 ioapics allowed + dict(machine=3D'pc-.*', device=3D'ioapic'), # Only 1 ioapics allowed + + # KVM-specific devices shouldn't be tried without accel=3Dkvm: + dict(accel=3D'(?!kvm).*', device=3D'kvmclock'), + dict(accel=3D'(?!kvm).*', device=3D'kvm-pci-assign'), + + # xen-specific machines and devices: + dict(accel=3D'(?!xen).*', machine=3D'xenfv'), # xenfv machine requires t= he xen accelerator + dict(accel=3D'(?!xen).*', machine=3D'xenpv'), + dict(accel=3D'(?!xen).*', device=3D'xen-platform'), # xen-platform devic= e requires the Xen accelerator + dict(device=3D'xen-pci-passthrough'), # Could not open '/sys/bus/pci/de= vices/0000:00:00.0/config': Permission denied + + # Some error messages that are common on multiple devices/machines: + dict(log=3Dr"No '[\w-]+' bus found for device '[\w-]+'"), + dict(log=3Dr"images* must be given with the 'pflash' parameter"), + dict(log=3Dr"(Guest|ROM|Flash|Kernel) image must be specified"), + dict(log=3Dr"[cC]ould not load [\w ]+ (BIOS|bios) '[\w-]+\.bin'"), + dict(log=3Dr"Couldn't find rom image '[\w-]+\.bin'"), + dict(log=3Dr"speed mismatch trying to attach usb device"), + dict(log=3Dr"Can't create a second ISA bus"), + dict(log=3Dr"duplicate fw_cfg file name"), + # sysbus-related error messages: most machines reject most dynamic sysbu= s devices: + dict(log=3Dr"Option '-device [\w.,-]+' cannot be handled by this machine= "), + dict(log=3Dr"Device [\w.,-]+ is not supported by this machine yet"), + dict(log=3Dr"Device [\w.,-]+ can not be dynamically instantiated"), + dict(log=3Dr"Platform Bus: Can not fit MMIO region of size "), + # other more specific errors we will ignore: + dict(device=3D'allwinner-a10', log=3D"Unsupported NIC model:"), + dict(device=3D'.*-spapr-cpu-core', log=3Dr"CPU core type should be"), + dict(log=3Dr"MSI(-X)? is not supported by interrupt controller"), + dict(log=3Dr"pxb-pcie? devices cannot reside on a PCIe? bus"), + dict(log=3Dr"Ignoring smp_cpus value"), + dict(log=3Dr"sd_init failed: Drive 'sd0' is already in use because it ha= s been automatically connected to another device"), + dict(log=3Dr"This CPU requires a smaller page size than the system is us= ing"), + dict(log=3Dr"MSI-X support is mandatory in the S390 architecture"), + dict(log=3Dr"rom check and register reset failed"), + dict(log=3Dr"Unable to initialize GIC, CPUState for CPU#0 not valid"), + dict(log=3Dr"Multiple VT220 operator consoles are not supported"), + dict(log=3Dr"core 0 already populated"), + + # KNOWN CRASHES: + # known crashes will generate error messages, but won't be fatal: + dict(exitcode=3D-6, log=3Dr"Device 'serial0' is in use", loglevel=3Dlogg= ing.ERROR), + dict(exitcode=3D-6, log=3Dr"spapr_rtas_register: Assertion .*rtas_table\= [token\]\.name.* failed", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"qemu_net_client_setup: Assertion `!peer->pee= r' failed", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr'RAMBlock "[\w.-]+" already registered', logl= evel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"find_ram_offset: Assertion `size !=3D 0' fai= led.", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"puv3_load_kernel: Assertion `kernel_filename= !=3D NULL' failed", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"add_cpreg_to_hashtable: code should not be r= eached", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"qemu_alloc_display: Assertion `surface->imag= e !=3D NULL' failed", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"Unexpected error in error_set_from_qdev_prop= _error", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"Object .* is not an instance of type spapr-m= achine", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"Object .* is not an instance of type generic= -pc-machine", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"Object .* is not an instance of type e500-cc= sr", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-6, log=3Dr"vmstate_register_with_alias_id: Assertion `!= se->compat || se->instance_id =3D=3D 0' failed", loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, device=3D'stm32f205-soc', loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, device=3D'xlnx,zynqmp', loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, device=3D'mips-cps', loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, device=3D'gus', loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, device=3D'a9mpcore_priv', loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, device=3D'isa-serial', loglevel=3Dlogging.ERROR), + dict(exitcode=3D-11, machine=3D'isapc', device=3D'.*-iommu', loglevel=3D= logging.ERROR), + + # other exitcode=3D1 failures not listed above will generate warnings: + dict(exitcode=3D1, loglevel=3Dlogging.WARN), + + # everything else (including SIGABRT and SIGSEGV) will be a fatal error: + dict(exitcode=3DNone, fatal=3DTrue, loglevel=3Dlogging.FATAL), +] + +def checkWhitelist(f): + """Look up whitelist entry for failure dictionary + + Returns index in ERROR_WHITELIST + """ + t =3D f['testcase'] + for i,wl in enumerate(ERROR_WHITELIST): + #dbg("whitelist entry: %r", wl) + if (wl.get('exitcode', 1) is None or + f['exitcode'] =3D=3D wl.get('exitcode', 1)) \ + and (not wl.has_key('machine') \ + or re.match(wl['machine'] +'$', t['machine'])) \ + and (not wl.has_key('accel') \ + or re.match(wl['accel'] +'$', t['accel'])) \ + and (not wl.has_key('device') \ + or re.match(wl['device'] +'$', t['device'])) \ + and (not wl.has_key('log') \ + or re.search(wl['log'], f['log'], re.MULTILINE)): + return i, wl + + raise Exception("this should never happen") + +def qemuOptsEscape(s): + return s.replace(",", ",,") + +def formatTestCase(t): + return ' '.join('%s=3D%s' % (k, v) for k,v in t.items()) + +def qomListTypeNames(vm, **kwargs): + """Run qom-list-types QMP command, return type names""" + types =3D vm.command('qom-list-types', **kwargs) + return [t['name'] for t in types] + +def infoQDM(vm): + """Parse 'info qdm' output""" + args =3D {'command-line': 'info qdm'} + devhelp =3D vm.command('human-monitor-command', **args) + for l in devhelp.split('\n'): + l =3D l.strip() + if l =3D=3D '' or l.endswith(':'): + continue + d =3D {'name': re.search(r'name "([^"]+)"', l).group(1), + 'no-user': (re.search(', no-user', l) is not None)} + #dbg('info qdm item: %r', d) + yield d + + +class QemuBinaryInfo: + def __init__(self, binary): + args =3D ['-S', '-machine', 'none,accel=3Dkvm:tcg'] + dbg("querying info for QEMU binary: %s", binary) + vm =3D QEMUQtestMachine(binary=3Dbinary, args=3Dargs) + vm.launch() + try: + + self.alldevs =3D set(qomListTypeNames(vm, implements=3D'device= ', abstract=3DFalse)) + # there's no way to query cannot_instantiate_with_device_add_y= et using QMP, + # so use 'info qdm': + self.no_user_devs =3D set([d['name'] for d in infoQDM(vm, ) if= d['no-user']]) + self.machines =3D list(m['name'] for m in vm.command('query-ma= chines')) + self.user_devs =3D self.alldevs.difference(self.no_user_devs) + self.kvm_available =3D vm.command('query-kvm')['enabled'] + finally: + vm.shutdown() + +BINARY_INFO =3D {} +def getBinaryInfo(binary): + if not BINARY_INFO.has_key(binary): + BINARY_INFO[binary] =3D QemuBinaryInfo(binary) + return BINARY_INFO[binary] + +def checkOneCase(testcase): + """Check one specific case + + Returns a dictionary containing failure information on error, + or None on success + """ + binary =3D testcase['binary'] + accel =3D testcase['accel'] + machine =3D testcase['machine'] + device =3D testcase['device'] + info =3D getBinaryInfo(binary) + + dbg("will test: %r", testcase) + + args =3D ['-S', '-machine', '%s,accel=3D%s' % (machine, accel), + '-device', qemuOptsEscape(device)] + cmdline =3D ' '.join([binary] + args) + dbg("will launch QEMU: %s", cmdline) + vm =3D QEMUQtestMachine(binary=3Dbinary, args=3Dargs) + + exception =3D None + try: + vm.launch() + logger.info("success: %s", cmdline) + except KeyboardInterrupt: + raise + except Exception,e: + exception =3D e + finally: + vm.shutdown() + ec =3D vm.exitcode() + log =3D vm.get_log() + + if exception is not None or ec !=3D 0: + f =3D dict(exception =3D e, + exitcode =3D ec, + log =3D log, + testcase =3D testcase, + cmdline=3Dcmdline) + return f + +def binariesToTest(args, testcase): + if args.qemu: + r =3D args.qemu + else: + r =3D glob.glob('./*-softmmu/qemu-system-*') + return r + +def accelsToTest(args, testcase): + if getBinaryInfo(testcase['binary']).kvm_available: + yield 'kvm' + yield 'tcg' + +def machinesToTest(args, testcase): + return getBinaryInfo(testcase['binary']).machines + +def devicesToTest(args, testcase): + return getBinaryInfo(testcase['binary']).user_devs + +TESTCASE_VARIABLES =3D [ + ('binary', binariesToTest), + ('accel', accelsToTest), + ('machine', machinesToTest), + ('device', devicesToTest), +] + +def genCases1(args, testcases, var, fn): + """Generate new testcases for one variable + + If an existing item already has a variable set, don't + generate new items and just return it directly. This + allows the "-t" command-line option to be used to choose + a specific test case. + """ + for testcase in testcases: + t =3D testcase.copy + if testcase.has_key(var): + yield testcase.copy() + else: + for i in fn(args, testcase): + t =3D testcase.copy() + t[var] =3D i + yield t + +def genCases(args, testcase): + """Generate test cases for all variables + + If args.random is set, generate a single item. Otherwise, + return all possible cases. + """ + cases =3D [testcase.copy()] + for var,fn in TESTCASE_VARIABLES: + dbg("var: %r, fn: %r", var, fn) + cases =3D genCases1(args, cases, var, fn) + if args.random: + cases =3D list(cases) + dbg("cases: %r", cases) + cases =3D [random.choice(cases)] + dbg("case: %r", cases) + return cases + +def genAllCases(args, testcase): + return genCases(args, testcase) + +def pickRandomCase(args, testcase): + cases =3D list(genCases(args, testcase)) + assert len(cases) =3D=3D 1 + return cases[0] + +def casesToTest(args, testcase): + if args.random: + cases =3D [pickRandomCase(args, testcase) for i in xrange(args.ran= dom)] + else: + cases =3D genAllCases(args, testcase) + if args.debug: + cases =3D list(cases) + dbg("%d test cases to test", len(cases)) + if args.shuffle: + cases =3D list(cases) + random.shuffle(cases) + return cases + +def logFailure(f, level): + t =3D f['testcase'] + logger.log(level, "failed: %s", formatTestCase(t)) + logger.log(level, "cmdline: %s", f['cmdline']) + for l in f['log'].strip().split('\n'): + logger.log(level, "log: %s", l) + logger.log(level, "exit code: %r", f['exitcode']) + +def main(): + parser =3D argparse.ArgumentParser(description=3D"QEMU -device crash t= est") + parser.add_argument('-t', metavar=3D'KEY=3DVALUE', nargs=3D'*', + help=3D"Limit test cases to KEY=3DVALUE", + dest=3D'testcases') + parser.add_argument('-d', '--debug', action=3D'store_true', + help=3D'debug output') + parser.add_argument('-v', '--verbose', action=3D'store_true', + help=3D'verbose output') + parser.add_argument('-r', '--random', type=3Dint, metavar=3D'COUNT', + help=3D'run a random sample of COUNT test cases', + default=3D0) + parser.add_argument('--shuffle', action=3D'store_true', + help=3D'Run test cases in random order') + parser.add_argument('--dry-run', action=3D'store_true', + help=3D"Don't run any tests, just generate list") + parser.add_argument('qemu', nargs=3D'*', metavar=3D'QEMU', + help=3D'QEMU binary to run') + args =3D parser.parse_args() + + if args.debug: + lvl =3D logging.DEBUG + elif args.verbose: + lvl =3D level=3Dlogging.INFO + else: + lvl =3D level=3Dlogging.WARN + logging.basicConfig(level=3Dlvl, format=3D'%(levelname)s: %(message)s') + + + interrupted =3D False + fatal_failures =3D [] + wl_stats =3D {} + + tc =3D {} + if args.testcases: + for t in args.testcases: + for kv in t.split(): + k,v =3D kv.split('=3D', 1) + tc[k] =3D v + + if len(binariesToTest(args, tc)) =3D=3D 0: + print >>sys.stderr, "No QEMU binary found" + parser.print_usage(sys.stderr) + return 1 + + for t in casesToTest(args, tc): + logger.info("test case: %s", formatTestCase(t)) + if args.dry_run: + continue + + try: + f =3D checkOneCase(t) + except KeyboardInterrupt: + interrupted =3D True + break + if f: + i,wl =3D checkWhitelist(f) + dbg("testcase: %r, whitelist match: %r", t, wl) + wl_stats.setdefault(i, []).append(f) + logFailure(f, wl.get('loglevel', logging.DEBUG)) + if wl.get('fatal'): + fatal_failures.append(f) + + # tell us about obsolete whitelist entries so we can clean it up after + # bugs are fixed: + if not interrupted and not args.random and not args.dry_run: + for i,wl in enumerate(ERROR_WHITELIST): + if not wl_stats.get(i): + logger.info("Obsolete whitelist entry? %r", wl) + + stats =3D sorted([(len(wl_stats[i]), i) for i in wl_stats]) + for count,i in stats: + dbg("whitelist entry stats: %d: %r", count, ERROR_WHITELIST[i]) + + if fatal_failures: + for f in fatal_failures: + t =3D f['testcase'] + logger.error("Fatal failure: %s", formatTestCase(t)) + logger.error("Fatal failures on some machine/device combinations") + return 1 + +if __name__ =3D=3D '__main__': + sys.exit(main()) --=20 2.11.0.259.g40922b1