From nobody Sun Sep 28 17:05:18 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1758010679; cv=none; d=zohomail.com; s=zohoarc; b=NMuVAlrim046QyhmTUNhJukiia4Zz4zUcJ6iwZIOqC3oslAjE2n5glVb1EZfSKP43e0N/tqGq1f8sQmWXo+22bwPBF7TUTDYP6mexYorgrTfLwWq1LcbDqzn1L3CRcsQV5JeF9ZyXNXkJA5gi1Pqo0TG/p7AxqllPFDpQhHy6ts= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1758010679; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=rda7XEQ3kq6g7pa3RFVVTTxi9DGLqiz9rOu/7eOSSWY=; b=K1iK25EEct/GjI0djdc4LkanXKLJddIHr+aSuFeqK9tC9d1WiU7nd1s7I/eVfFN/znhegcXNG8090WvnCttYuJEIOtAVwib/eioSf09a5bqxNS6cABR2Lbl/j/hu7Z9gxYKzL9vhdx+zZcNE6AX7tgL8s1N9z94qFce8mvNMxwk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1758010679417430.13307486333906; Tue, 16 Sep 2025 01:17:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uyQsU-0003tY-Jm; Tue, 16 Sep 2025 04:17:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uyQsK-0003oQ-Jz for qemu-devel@nongnu.org; Tue, 16 Sep 2025 04:17:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uyQsG-00045F-M9 for qemu-devel@nongnu.org; Tue, 16 Sep 2025 04:17:20 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-348-ooV5XtoFMr6-Qj3ZwUl4xg-1; Tue, 16 Sep 2025 04:17:11 -0400 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6165F1800577; Tue, 16 Sep 2025 08:17:10 +0000 (UTC) Received: from toolbx.redhat.com (unknown [10.42.28.153]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 311461800446; Tue, 16 Sep 2025 08:17:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1758010634; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rda7XEQ3kq6g7pa3RFVVTTxi9DGLqiz9rOu/7eOSSWY=; b=XjbIhNFGJrgJMgWLpfUaiNv5cEnp9V0FP4jfBBR1Iaz2sIRnBFSqz8WP3RuAlbDv35mScl ud2FT/yNujzZQT7pgV7Ac/B3jbpm8LrHSQLza1nvJkLhgIR+uIaStggO/iHF2PmL6F8w7l Fro0qPeHOaZDxdTl6DHoxzcRbhGAXhw= X-MC-Unique: ooV5XtoFMr6-Qj3ZwUl4xg-1 X-Mimecast-MFC-AGG-ID: ooV5XtoFMr6-Qj3ZwUl4xg_1758010630 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Thomas Huth , Michael Roth , Stefan Hajnoczi , Paolo Bonzini , Mads Ynddal , =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v5 7/8] qapi: switch to use QEMU_TEST_REGENERATE env var Date: Tue, 16 Sep 2025 09:16:37 +0100 Message-ID: <20250916081638.764020-8-berrange@redhat.com> In-Reply-To: <20250916081638.764020-1-berrange@redhat.com> References: <20250916081638.764020-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.035, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1758010681113116600 The QAPI_TEST_UPDATE env var can be set when running the QAPI schema tests to regenerate the reference output. For consistent naming with the tracetool test, change the env var name to QEMU_TEST_REGENERATE. The test is modified to provide a hint about use of the new env var and it is also added to the developer documentation.document its us= age. Reviewed-by: Stefan Hajnoczi Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Markus Armbruster --- docs/devel/testing/main.rst | 12 ++++++++++++ tests/qapi-schema/test-qapi.py | 7 +++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst index 11f05c0006..0662766b5c 100644 --- a/docs/devel/testing/main.rst +++ b/docs/devel/testing/main.rst @@ -178,6 +178,18 @@ parser (either fixing a bug or extending/modifying the= syntax). To do this: =20 ``qapi-schema +=3D foo.json`` =20 +The reference output can be automatically updated to match the latest QAPI +code generator by running the tests with the QEMU_TEST_REGENERATE environm= ent +variable set. + +.. code:: + + QEMU_TEST_REGENERATE=3D1 make check-qapi-schema + +The resulting changes must be reviewed by the author to ensure they match +the intended results before adding the updated reference output to the +same commit that alters the generator code. + .. _tracetool-tests: =20 Tracetool tests diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py index 4be930228c..cf7fb8a6df 100755 --- a/tests/qapi-schema/test-qapi.py +++ b/tests/qapi-schema/test-qapi.py @@ -165,7 +165,7 @@ def test_and_diff(test_name, dir_name, update): if actual_out =3D=3D expected_out and actual_err =3D=3D expected_err: return 0 =20 - print("%s %s" % (test_name, 'UPDATE' if update else 'FAIL'), + print("%s: %s" % (test_name, 'UPDATE' if update else 'FAIL'), file=3Dsys.stderr) out_diff =3D difflib.unified_diff(expected_out, actual_out, outfp.name) err_diff =3D difflib.unified_diff(expected_err, actual_err, errfp.name) @@ -173,6 +173,9 @@ def test_and_diff(test_name, dir_name, update): sys.stdout.writelines(err_diff) =20 if not update: + print(("\n%s: set QEMU_TEST_REGENERATE=3D1 to recreate reference o= utput" + + "if the QAPI schema generator was intentionally changed") %= test_name, + file=3Dsys.stderr) return 1 =20 try: @@ -197,7 +200,7 @@ def main(argv): parser.add_argument('-d', '--dir', action=3D'store', default=3D'', help=3D"directory containing tests") parser.add_argument('-u', '--update', action=3D'store_true', - default=3D'QAPI_TEST_UPDATE' in os.environ, + default=3D'QEMU_TEST_REGENERATE' in os.environ, help=3D"update expected test results") parser.add_argument('tests', nargs=3D'*', metavar=3D'TEST', action=3D'= store') args =3D parser.parse_args() --=20 2.50.1