From nobody Wed Feb 11 02:09:05 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529425443097487.105032943986; Tue, 19 Jun 2018 09:24:03 -0700 (PDT) Received: from localhost ([::1]:43597 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVJQI-00059N-8A for importer@patchew.org; Tue, 19 Jun 2018 12:24:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46058) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVJ50-0004fg-Sx for qemu-devel@nongnu.org; Tue, 19 Jun 2018 12:02:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVJ4x-0007f7-Mq for qemu-devel@nongnu.org; Tue, 19 Jun 2018 12:02:02 -0400 Received: from mail-wm0-x22b.google.com ([2a00:1450:400c:c09::22b]:53386) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVJ4x-0007eQ-DM for qemu-devel@nongnu.org; Tue, 19 Jun 2018 12:01:59 -0400 Received: by mail-wm0-x22b.google.com with SMTP id x6-v6so1257246wmc.3 for ; Tue, 19 Jun 2018 09:01:59 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id z16-v6sm58140wro.41.2018.06.19.09.01.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 19 Jun 2018 09:01:55 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 963273E0FC0; Tue, 19 Jun 2018 16:44:39 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BJdJzDkjcl0zIXjcW2r0b/g6OgOf6EdG+dfqirQG5js=; b=JPn1CntZxeMuDZDe9oDUn/70Vm2y3rArqIj5MAl8qBsvIlsp2xznerjz3WUXLjb6Ko yNnXbpR5sByqqZeR5cAEZSKTLELav3MXLE8ChlAmTEfytfC5vKWKB9AwpArACSuZ6hLX WO6KuXAc2lv+KT/Ub5zVZXmzY8Iff19s6z2Z8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BJdJzDkjcl0zIXjcW2r0b/g6OgOf6EdG+dfqirQG5js=; b=b6jt01Zx63EiTaNAxWBlzMFDMve3Xam7Z7xfRO4pV7B/GZVtYD1fhaCdLXk6+gGakd Szt0CeWBOp6RwdFaLLs8grorQXMSXRqv2kLMGsVuupyyhTuNF78bxLuikldVtPy9z5lF daO3Eth36lsU+KciUlKApfyrKaxwzWEsKm/hteLrhbuzZhNyhr/tMtV5LZeDvWUdm5hP mpZL1yKJnZQD6iAcQoVVzxioTPOKcXMsyDQmv1jH1iBqzE2fQ1S5HaPyz+ioCuJdGB8U 2h5TU1utHQrvnVyTVRUOVS0rrjNJMdcjeJZ5/hh4fsrEPfZ7a+NXruNUFRO39fMQI+im I0iA== X-Gm-Message-State: APt69E1ZDiP/+rkxdR9hUrPVWQFG9QQWxHaA1YkPiJwBnFvGO4SP+itd nN1Etdoslr3lRaaEulgU7mRoliQF8gA= X-Google-Smtp-Source: ADUXVKJj52JtyGZ9FwOiDm7he4g8whFmHx6VEYMKSWgd5wHfIKkpmRwfQIjLhUVqz41Jz0rTyF8Bnw== X-Received: by 2002:a1c:8cc8:: with SMTP id o191-v6mr12464357wmd.75.1529424118222; Tue, 19 Jun 2018 09:01:58 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: peter.maydell@linaro.org Date: Tue, 19 Jun 2018 16:44:25 +0100 Message-Id: <20180619154435.18898-47-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180619154435.18898-1-alex.bennee@linaro.org> References: <20180619154435.18898-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22b Subject: [Qemu-devel] [PULL 46/56] tests/tcg: add run, diff, and skip helper macros X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: agraf@suse.de, Eduardo Habkost , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, qemu-devel@nongnu.org, f4bug@amsat.org, cota@braap.org, "open list:ARM" , Paolo Bonzini , famz@redhat.com, aurelien@aurel32.net, Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 As we aren't using the default runners for all the test cases it is easy to miss out things like timeouts. To help with this we add some helpers and use them so we only need to make core changes in one place. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- v5 - new for v5 v5.1 - diff-test->diff-out v7 - mention $(TARGET_NAME) when skipping diff --git a/tests/tcg/Makefile b/tests/tcg/Makefile index e7dbcdb5bf..bf06415390 100644 --- a/tests/tcg/Makefile +++ b/tests/tcg/Makefile @@ -34,6 +34,15 @@ =20 quiet-command =3D $(if $(V),$1,$(if $(2),@printf " %-7s %s\n" $2 $3 && $1= , @$1)) =20 +# $1 =3D test name, $2 =3D cmd, $3 =3D desc +run-test =3D $(call quiet-command, timeout $(TIMEOUT) $2 > $1.out,"TEST",$= 3) + +# $1 =3D test name, $2 =3D reference +diff-out =3D $(call quiet-command, diff -u $1.out $2 | head -n 10,"DIFF","= $1.out with $2") + +# $1 =3D test name, $2 =3D reason +skip-test =3D @printf " SKIPPED %s on $(TARGET_NAME) because %s\n" $1 $2 + # Tests we are building TESTS=3D =20 @@ -84,9 +93,7 @@ RUN_TESTS=3D$(patsubst %,run-%, $(TESTS)) RUN_TESTS+=3D$(EXTRA_RUNS) =20 run-%: % - $(call quiet-command, \ - timeout $(TIMEOUT) $(QEMU) $< > $<.out, \ - "TEST", "$< on $(TARGET_NAME)") + $(call run-test, $<, $(QEMU) $<, "$< on $(TARGET_NAME)") =20 .PHONY: run run: $(RUN_TESTS) diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile= .target index 7dba32138d..08c45b8470 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -13,7 +13,5 @@ TESTS:=3D$(AARCH64_TESTS) fcvt: LDFLAGS+=3D-lm =20 run-fcvt: fcvt - $(call quiet-command, \ - $(QEMU) $< > fcvt.out && \ - diff -u $(AARCH64_SRC)/fcvt.ref fcvt.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,$<,$(QEMU) $<, "$< on $(TARGET_NAME)") + $(call diff-out,$<,$(AARCH64_SRC)/fcvt.ref) diff --git a/tests/tcg/arm/Makefile.target b/tests/tcg/arm/Makefile.target index 14b5435fc6..aa4e4e3782 100644 --- a/tests/tcg/arm/Makefile.target +++ b/tests/tcg/arm/Makefile.target @@ -24,10 +24,8 @@ fcvt: LDFLAGS+=3D-lm # fcvt: CFLAGS+=3D-march=3Darmv8.2-a+fp16 -mfpu=3Dneon-fp-armv8 =20 run-fcvt: fcvt - $(call quiet-command, \ - $(QEMU) $< > fcvt.out && \ - diff -u $(ARM_SRC)/fcvt.ref fcvt.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,fcvt,$(QEMU) $<,"$< on $(TARGET_NAME)") + $(call diff-out,fcvt,$(ARM_SRC)/fcvt.ref) endif =20 # On ARM Linux only supports 4k pages diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target index cd173363ee..97b7c23cf1 100644 --- a/tests/tcg/i386/Makefile.target +++ b/tests/tcg/i386/Makefile.target @@ -31,7 +31,7 @@ test-i386: test-i386.c test-i386-code16.S test-i386-vm86.= S test-i386.h test-i386 =20 # Specialist test runners run-runcom: runcom pi_10.com - $(call quiet-command, $(QEMU) ./runcom $(I386_SRC)/pi_10.com > runcom.out= , "TEST", "$< on $(TARGET_NAME)") + $(call run-test,$<,$(QEMU) ./runcom $(I386_SRC)/pi_10.com,"$< on $(TARGET= _NAME)") =20 ifeq ($(SPEED), slow) =20 @@ -40,13 +40,11 @@ test-i386-fprem.ref: test-i386-fprem =20 run-test-i386-fprem: TIMEOUT=3D60 run-test-i386-fprem: test-i386-fprem - $(call quiet-command, \ - $(QEMU) $< > $<.out && \ - diff -u $(I386_SRC)/$<.ref $<.out, \ - "TEST", "$< (default) on $(TARGET_NAME)") + $(call run-test,test-i386-fprem, $(QEMU) $<,"$< on $(TARGET_NAME)") + $(call diff-out,test-i386-fprem, $(I386_SRC)/$<.ref) else run-test-i386-fprem: test-i386-fprem - $(call quiet-command, /bin/true, "SLOW TEST", "$< SKIPPED on $(TARGET_NAM= E)") + $(call skip-test, $<, "SLOW") endif =20 # On i386 and x86_64 Linux only supports 4k pages (large pages are a diffe= rent hack) diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Make= file.target index 90e45a881e..b77084c146 100644 --- a/tests/tcg/multiarch/Makefile.target +++ b/tests/tcg/multiarch/Makefile.target @@ -27,10 +27,10 @@ testthread: LDFLAGS+=3D-lpthread =20 # default case (host page size) run-test-mmap: test-mmap - $(call quiet-command, $(QEMU) $< > test-mmap.out, "TEST", \ + $(call run-test, test-mmap, $(QEMU) $<, \ "$< (default) on $(TARGET_NAME)") =20 # additional page sizes (defined by each architecture adding to EXTRA_RUNS) run-test-mmap-%: test-mmap - $(call quiet-command, $(QEMU) -p $* $< > test-mmap-$*.out, "TEST", \ + $(call run-test, test-mmap-$*, $(QEMU) -p $* $<,\ "$< ($* byte pages) on $(TARGET_NAME)") --=20 2.17.1