From nobody Thu May 2 07:51:28 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1545067650301225.824802300967; Mon, 17 Dec 2018 09:27:30 -0800 (PST) Received: from localhost ([::1]:48151 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gYwfw-0006ya-QI for importer@patchew.org; Mon, 17 Dec 2018 12:27:28 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41943) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gYwf5-0006av-B6 for qemu-devel@nongnu.org; Mon, 17 Dec 2018 12:26:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gYwf0-0002A5-Ij for qemu-devel@nongnu.org; Mon, 17 Dec 2018 12:26:35 -0500 Received: from relay.sw.ru ([185.231.240.75]:39458) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gYwf0-00028t-Al; Mon, 17 Dec 2018 12:26:30 -0500 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gYwex-0001S9-8e; Mon, 17 Dec 2018 20:26:27 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 17 Dec 2018 20:26:26 +0300 Message-Id: <20181217172626.34501-1-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH] iotests.py: rework log and qmp_log X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There are several problems to be solved: 1. log() may try to apply filters (which should be str -> str functions) to objects, which is not correct. Let's instead do json.dumps first and then apply filters. 2. Let's drop trailing whitespaces, if there are any. Hopefully, now only two tests have trailing whitespaces, both bash-based: 130, 181. 3. qmp_log() do json.dumps() separately and call log(), when log() can print objects too. It's obviously a duplication, let's instead pass objects from qmp_log to log 4. qmp_log may produce lines exceeding 998 characters, which is unfriendly to git send-email. To fix it, let's use pretty json output, by setting indent parameter of json.dumps. Note that with that parameter, dumps may produce lines with trailing whitespaces, so [2.] becomes very necessary. It may be a good thing (keeping in mind, that putting large objects to log() will lead to the same problem), to prettify all json in tests output, but it would be very large patch, hard to review. So, let's keep qmp_log_depr for old behavior and don't prettify log() output by default. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/206 | 4 ++-- tests/qemu-iotests/207 | 2 +- tests/qemu-iotests/210 | 12 ++++++------ tests/qemu-iotests/211 | 4 ++-- tests/qemu-iotests/212 | 4 ++-- tests/qemu-iotests/213 | 4 ++-- tests/qemu-iotests/iotests.py | 28 ++++++++++++++++++++-------- 7 files changed, 35 insertions(+), 23 deletions(-) diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206 index 128c334c7c..cc5b5098ce 100755 --- a/tests/qemu-iotests/206 +++ b/tests/qemu-iotests/206 @@ -26,7 +26,7 @@ from iotests import imgfmt iotests.verify_image_format(supported_fmts=3D['qcow2']) =20 def blockdev_create(vm, options): - result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions) + result =3D vm.qmp_log_depr('blockdev-create', job_id=3D'job0', options= =3Doptions) =20 if 'return' in result: assert result['return'] =3D=3D {} @@ -52,7 +52,7 @@ with iotests.FilePath('t.qcow2') as disk_path, \ 'filename': disk_path, 'size': 0 }) =20 - vm.qmp_log('blockdev-add', driver=3D'file', filename=3Ddisk_path, + vm.qmp_log_depr('blockdev-add', driver=3D'file', filename=3Ddisk_path, node_name=3D'imgfile') =20 blockdev_create(vm, { 'driver': imgfmt, diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207 index c617ee7453..f5b74e5bfb 100755 --- a/tests/qemu-iotests/207 +++ b/tests/qemu-iotests/207 @@ -31,7 +31,7 @@ def filter_hash(msg): return re.sub('"hash": "[0-9a-f]+"', '"hash": HASH', msg) =20 def blockdev_create(vm, options): - result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions, + result =3D vm.qmp_log_depr('blockdev-create', job_id=3D'job0', options= =3Doptions, filters=3D[iotests.filter_testfiles, filter_hash]) =20 if 'return' in result: diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210 index d142841e2b..b06232861e 100755 --- a/tests/qemu-iotests/210 +++ b/tests/qemu-iotests/210 @@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=3D['luks']) iotests.verify_protocol(supported=3D['file']) =20 def blockdev_create(vm, options): - result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions) + result =3D vm.qmp_log_depr('blockdev-create', job_id=3D'job0', options= =3Doptions) =20 if 'return' in result: assert result['return'] =3D=3D {} @@ -52,7 +52,7 @@ with iotests.FilePath('t.luks') as disk_path, \ 'filename': disk_path, 'size': 0 }) =20 - vm.qmp_log('blockdev-add', driver=3D'file', filename=3Ddisk_path, + vm.qmp_log_depr('blockdev-add', driver=3D'file', filename=3Ddisk_path, node_name=3D'imgfile') =20 blockdev_create(vm, { 'driver': imgfmt, @@ -170,10 +170,10 @@ with iotests.FilePath('t.luks') as disk_path, \ =20 vm.add_blockdev('driver=3Dluks,file=3Dnode0,key-secret=3Dkeysec0,node-= name=3Dnode1') vm.launch() - vm.qmp_log('block_resize', node_name=3D'node1', size=3D922337203685477= 5296) - vm.qmp_log('block_resize', node_name=3D'node1', size=3D922337203685477= 5808) - vm.qmp_log('block_resize', node_name=3D'node1', size=3D184467440737095= 51104) - vm.qmp_log('block_resize', node_name=3D'node1', size=3D-92233720368547= 75808) + vm.qmp_log_depr('block_resize', node_name=3D'node1', size=3D9223372036= 854775296) + vm.qmp_log_depr('block_resize', node_name=3D'node1', size=3D9223372036= 854775808) + vm.qmp_log_depr('block_resize', node_name=3D'node1', size=3D1844674407= 3709551104) + vm.qmp_log_depr('block_resize', node_name=3D'node1', size=3D-922337203= 6854775808) vm.shutdown() =20 # TODO Proper support for images to be used with imgopts and/or protoc= ols diff --git a/tests/qemu-iotests/211 b/tests/qemu-iotests/211 index 7b7985db6c..c791499659 100755 --- a/tests/qemu-iotests/211 +++ b/tests/qemu-iotests/211 @@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=3D['vdi']) iotests.verify_protocol(supported=3D['file']) =20 def blockdev_create(vm, options): - result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions) + result =3D vm.qmp_log_depr('blockdev-create', job_id=3D'job0', options= =3Doptions) =20 if 'return' in result: assert result['return'] =3D=3D {} @@ -50,7 +50,7 @@ with iotests.FilePath('t.vdi') as disk_path, \ 'filename': disk_path, 'size': 0 }) =20 - vm.qmp_log('blockdev-add', driver=3D'file', filename=3Ddisk_path, + vm.qmp_log_depr('blockdev-add', driver=3D'file', filename=3Ddisk_path, node_name=3D'imgfile') =20 blockdev_create(vm, { 'driver': imgfmt, diff --git a/tests/qemu-iotests/212 b/tests/qemu-iotests/212 index 95c8810d83..6fbe2c278d 100755 --- a/tests/qemu-iotests/212 +++ b/tests/qemu-iotests/212 @@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=3D['parallels'= ]) iotests.verify_protocol(supported=3D['file']) =20 def blockdev_create(vm, options): - result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions) + result =3D vm.qmp_log_depr('blockdev-create', job_id=3D'job0', options= =3Doptions) =20 if 'return' in result: assert result['return'] =3D=3D {} @@ -50,7 +50,7 @@ with iotests.FilePath('t.parallels') as disk_path, \ 'filename': disk_path, 'size': 0 }) =20 - vm.qmp_log('blockdev-add', driver=3D'file', filename=3Ddisk_path, + vm.qmp_log_depr('blockdev-add', driver=3D'file', filename=3Ddisk_path, node_name=3D'imgfile') =20 blockdev_create(vm, { 'driver': imgfmt, diff --git a/tests/qemu-iotests/213 b/tests/qemu-iotests/213 index 4054439e3c..8f854417e3 100755 --- a/tests/qemu-iotests/213 +++ b/tests/qemu-iotests/213 @@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=3D['vhdx']) iotests.verify_protocol(supported=3D['file']) =20 def blockdev_create(vm, options): - result =3D vm.qmp_log('blockdev-create', job_id=3D'job0', options=3Dop= tions) + result =3D vm.qmp_log_depr('blockdev-create', job_id=3D'job0', options= =3Doptions) =20 if 'return' in result: assert result['return'] =3D=3D {} @@ -50,7 +50,7 @@ with iotests.FilePath('t.vhdx') as disk_path, \ 'filename': disk_path, 'size': 0 }) =20 - vm.qmp_log('blockdev-add', driver=3D'file', filename=3Ddisk_path, + vm.qmp_log_depr('blockdev-add', driver=3D'file', filename=3Ddisk_path, node_name=3D'imgfile') =20 blockdev_create(vm, { 'driver': imgfmt, diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index d537538ba0..8f1ea23ef7 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -251,13 +251,17 @@ def filter_img_info(output, filename): lines.append(line) return '\n'.join(lines) =20 -def log(msg, filters=3D[]): +def log(msg, filters=3D[], indent=3DNone): + if type(msg) is dict or type(msg) is list: + msg =3D json.dumps(msg, sort_keys=3DTrue, indent=3Dindent) + for flt in filters: msg =3D flt(msg) - if type(msg) is dict or type(msg) is list: - print(json.dumps(msg, sort_keys=3DTrue)) - else: - print(msg) + + # drop trailing whitespaces + msg =3D re.sub(r'[ \t]+$', '', msg, flags=3Dre.MULTILINE) + + print(msg) =20 class Timeout: def __init__(self, seconds, errmsg =3D "Timeout"): @@ -444,7 +448,9 @@ class VM(qtest.QEMUQtestMachine): result.append(filter_qmp_event(ev)) return result =20 - def qmp_log(self, cmd, filters=3D[filter_testfiles], **kwargs): + # qmp_log_depr is deprecated. + # TODO: replace all it's usage cases with qmp_log, and drop the defini= tion. + def qmp_log_depr(self, cmd, filters=3D[filter_testfiles], **kwargs): logmsg =3D '{"execute": "%s", "arguments": %s}' % \ (cmd, json.dumps(kwargs, sort_keys=3DTrue)) log(logmsg, filters) @@ -452,6 +458,12 @@ class VM(qtest.QEMUQtestMachine): log(json.dumps(result, sort_keys=3DTrue), filters) return result =20 + def qmp_log(self, cmd, filters=3D[filter_testfiles], **kwargs): + log({'execute': cmd, 'arguments': kwargs}, filters, indent=3D2) + result =3D self.qmp(cmd, **kwargs) + log(result, filters, indent=3D2) + return result + def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse): while True: for ev in self.get_qmp_events_filtered(wait=3DTrue): @@ -463,9 +475,9 @@ class VM(qtest.QEMUQtestMachine): if j['id'] =3D=3D job: log('Job failed: %s' % (j['error'])) elif status =3D=3D 'pending' and not auto_finalize: - self.qmp_log('job-finalize', id=3Djob) + self.qmp_log_depr('job-finalize', id=3Djob) elif status =3D=3D 'concluded' and not auto_dismiss: - self.qmp_log('job-dismiss', id=3Djob) + self.qmp_log_depr('job-dismiss', id=3Djob) elif status =3D=3D 'null': return else: --=20 2.18.0