From nobody Sat May 4 23:46:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529501311701454.7314958015752; Wed, 20 Jun 2018 06:28:31 -0700 (PDT) Received: from localhost ([::1]:49675 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd9u-0006mU-WF for importer@patchew.org; Wed, 20 Jun 2018 09:28:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54992) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd2N-0007yA-Dy for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVd2K-0002kD-1s for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:39 -0400 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]:50499) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVd2J-0002jy-Pz for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:35 -0400 Received: by mail-wm0-x244.google.com with SMTP id e16-v6so5981715wmd.0 for ; Wed, 20 Jun 2018 06:20:35 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id b8-v6sm3103443wrp.50.2018.06.20.06.20.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 06:20:34 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id F0DF63E00A7; Wed, 20 Jun 2018 14:20:32 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wN6XNRgedUtJrvEhyG6whnN3+CYpPICCDRGCe6ZVR1A=; b=KjVd/1CmetSEHqdlwRg4woAfBqUMlaoTFbp8h8iQFxs90J7qCizN2TPq5SrsDkuy8U NvcdvBg7D4NmkTTFMVNhrLXzKzPDW/X+2cGcaCD2mNJ9EGhUgQuBnL8OVoTb9Zoq3ts6 2XZ0exQ4D9GkvKQTNxXM+XX/gh9na1h4UNrTo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wN6XNRgedUtJrvEhyG6whnN3+CYpPICCDRGCe6ZVR1A=; b=CPwDNqkFsJ3ngoPbi9K2GZfLwFz5A+Dj23D4PMTJ4ag19zS/eZ3ZSJALrNJ0+KbxKK CQUDClA64Gzkr500PpgChoCRVYFGcQKQSBu+OdzjGgKAG10Jjg7Sp4tpJBI3bjyvvxFC nSmPmD0JxSZpXBwl/PBUjLeQP6PqKspDc4flZEqhufX95MCW82c79PjMIUJs8psLoOsN 32ARZI0ybHEPwF2qByQV7lZ3NWwbbbld8PNLF4STKqD1ZfGhlRgtyJCEARPJExdmjgyw KxMZE7y1kXTsQBGn8monZ/75Mpe6HT2pkP79UCDfLfW2TgSbvKLKRl7wviLCtVV+v4Ao mxSg== X-Gm-Message-State: APt69E3dJ3Nmfw6hVa2fg+pisWVLV/NrcUZCr0pAGLqky5M+4DnraiCm os12SPZd/zzazlsKheGprYtheg== X-Google-Smtp-Source: ADUXVKKD6eT18uruOHvE+kVjSgorsFGMS1tmKhaq5qnVuh0DN2urjiUYSJnMA+xaOcnzqseX+deyzw== X-Received: by 2002:a1c:9bd0:: with SMTP id d199-v6mr1780948wme.16.1529500834754; Wed, 20 Jun 2018 06:20:34 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: cota@braap.org, famz@redhat.com, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Wed, 20 Jun 2018 14:20:28 +0100 Message-Id: <20180620132032.12952-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180620132032.12952-1-alex.bennee@linaro.org> References: <20180620132032.12952-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::244 Subject: [Qemu-devel] [RFC PATCH 1/5] build-system: remove per-test GCOV reporting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 I'm not entirely sure who's using this information and certainly in a CI environment it just washes over as additional noise. Later patches will provide new reporting options so a user who wants to analyse individual tests will be able to use that to get the information. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- docs/devel/testing.rst | 11 +++++------ tests/Makefile.include | 10 ---------- 2 files changed, 5 insertions(+), 16 deletions(-) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index f33e5a8423..66ef219f69 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -158,12 +158,11 @@ rarely used. See "QEMU iotests" section below for mor= e information. GCC gcov support ---------------- =20 -``gcov`` is a GCC tool to analyze the testing coverage by instrumenting the -tested code. To use it, configure QEMU with ``--enable-gcov`` option and b= uild. -Then run ``make check`` as usual. There will be additional ``gcov`` output= as -the testing goes on, showing the test coverage percentage numbers per anal= yzed -source file. More detailed reports can be obtained by running ``gcov`` com= mand -on the output files under ``$build_dir/tests/``, please read the ``gcov`` +``gcov`` is a GCC tool to analyze the testing coverage by +instrumenting the tested code. To use it, configure QEMU with +``--enable-gcov`` option and build. Then run ``make check`` as usual. +Reports can be obtained by running ``gcov`` command on the output +files under ``$build_dir/tests/``, please read the ``gcov`` documentation for more information. =20 QEMU iotests diff --git a/tests/Makefile.include b/tests/Makefile.include index ca91da26cb..55d54bd180 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -891,26 +891,16 @@ GCOV_OPTIONS =3D -n $(if $(V),-f,) =20 .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS)) $(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-so= ftmmu $(check-qtest-y) - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,) $(call quiet-command,QTEST_QEMU_BINARY=3D$*-softmmu/qemu-system-$* \ QTEST_QEMU_IMG=3Dqemu-img$(EXESUF) \ MALLOC_PERTURB_=3D$${MALLOC_PERTURB_:-$$(( $${RANDOM:-0} % 255 + 1))} \ gtester $(GTESTER_OPTIONS) -m=3D$(SPEED) $(check-qtest-$*-y) $(check-qte= st-generic-y),"GTESTER","$@") - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$*-y) $(gcov-files-generic-y);= do \ - echo Gcov report for $$f:;\ - $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \ - done,) =20 .PHONY: $(patsubst %, check-%, $(check-unit-y) $(check-speed-y)) $(patsubst %, check-%, $(check-unit-y) $(check-speed-y)): check-%: % - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,) $(call quiet-command, \ MALLOC_PERTURB_=3D$${MALLOC_PERTURB_:-$$(( $${RANDOM:-0} % 255 + 1))} \ gtester $(GTESTER_OPTIONS) -m=3D$(SPEED) $*,"GTESTER","$*") - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$(subst tests/,,$*)-y) $(gcov-= files-generic-y); do \ - echo Gcov report for $$f:;\ - $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \ - done,) =20 # gtester tests with XML output =20 --=20 2.17.1 From nobody Sat May 4 23:46:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529500971263622.2999515647541; Wed, 20 Jun 2018 06:22:51 -0700 (PDT) Received: from localhost ([::1]:49633 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd4S-0000nz-61 for importer@patchew.org; Wed, 20 Jun 2018 09:22:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54991) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd2N-0007y9-DR for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVd2K-0002kU-Qa for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:39 -0400 Received: from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241]:52981) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVd2K-0002k8-K8 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:36 -0400 Received: by mail-wm0-x241.google.com with SMTP id p126-v6so5990738wmb.2 for ; Wed, 20 Jun 2018 06:20:36 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id b204-v6sm2595773wmh.22.2018.06.20.06.20.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 06:20:34 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 24D173E00D2; Wed, 20 Jun 2018 14:20:33 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ccjdWaqoluWFkOn2xFKw3BI4GzIQoWCqGahi56R6ic8=; b=f1lyPuDSZZnH27jI36niojCOeoZH1OSQALnvIaZSATL57nTljW+ktyf20qQ3o9Amf8 phED64G85fI3KkvVidUx1w+NjOLBwOevJ93Uu40pLr/k+EOGEhSGfPkHhWsZda5FkbCD rD1eKBQSc5AsWKmjbbDQSMiEXnwndnZ5w/ZFA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ccjdWaqoluWFkOn2xFKw3BI4GzIQoWCqGahi56R6ic8=; b=V3cyc4QLfLkNvVcmnnq++ItWeirlfSxe2uDJWmjgBEjZWWy+KOAv4C6rH3bmTENjJ+ jNzHOqgUuRHpymW2Ij/bkNEGiXi2USAPp14CyNuIkAi4UmS2rz40HLce4Tpmh7gU8vEy Uvp4gqnV7HI+XqBgyHOdEkq3zk489u5AGAzQVm0sUV7Aw6E8svC4EY3ETTu/URI3nIib z1vkkxXOWwdZNfr+9HYLwe2q5/1l5jNbcFTmnE5iH9oDAcqBk2Ff+5CZb729hmo4DHzj +oz1o292BwD550ptempbrXUiDbrtxB00DsBx9tefsd0OQR2ILXY1kRhWn6unygTCDy87 R0xg== X-Gm-Message-State: APt69E1i+Am+e2Hc8jdVF1H+qTUycVLF+lDaV1K1+1uY2SRTjkPnTpkC nhYX0NHmLdc9D7yq0OHIYoWmPQ== X-Google-Smtp-Source: ADUXVKLXCjJJgBSz29mqNuVV7vvqTAC2GUeIMo821pbziXbCAj3HMU5wBB1H1y4K0eN/mS8ruCT4IA== X-Received: by 2002:a1c:f415:: with SMTP id z21-v6mr1666367wma.80.1529500835581; Wed, 20 Jun 2018 06:20:35 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: cota@braap.org, famz@redhat.com, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Wed, 20 Jun 2018 14:20:29 +0100 Message-Id: <20180620132032.12952-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180620132032.12952-1-alex.bennee@linaro.org> References: <20180620132032.12952-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::241 Subject: [Qemu-devel] [RFC PATCH 2/5] .gitignore: add .gcov files X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 These are temporary files generated on gcov runs and shouldn't be included in the source tree. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 9da3b3e626..5668d02782 100644 --- a/.gitignore +++ b/.gitignore @@ -155,6 +155,7 @@ .sdk *.gcda *.gcno +*.gcov /pc-bios/bios-pq/status /pc-bios/vgabios-pq/status /pc-bios/optionrom/linuxboot.asm --=20 2.17.1 From nobody Sat May 4 23:46:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529501396638656.7630933008064; Wed, 20 Jun 2018 06:29:56 -0700 (PDT) Received: from localhost ([::1]:49685 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVdBI-00080O-0F for importer@patchew.org; Wed, 20 Jun 2018 09:29:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54993) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd2N-0007yB-Dw for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVd2L-0002kw-NE for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:39 -0400 Received: from mail-wm0-x22f.google.com ([2a00:1450:400c:c09::22f]:37754) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVd2L-0002kS-Fb for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:37 -0400 Received: by mail-wm0-x22f.google.com with SMTP id r125-v6so6890114wmg.2 for ; Wed, 20 Jun 2018 06:20:37 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id c201-v6sm4080836wmh.18.2018.06.20.06.20.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 06:20:34 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 48D283E08C1; Wed, 20 Jun 2018 14:20:33 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gIgbUK2fBQGW7ekPtBlXAL4um1NhJ9HA4ZfsnksaiAs=; b=WNAJ/dmcl1p0/E9TegAVb+m7IIQH/CKHpavZjlhYeGEIWpL8Omy41UczKMzf1RawOt //OPfPEcno4jEBiU8GQD4vSIy7MopCwITIKejTECL4e6uvAuou+e2uWUMFyLQ72BHPsb 2pSgMwBHywdaLyHeP7EOHFqWitUorJF5e/ChM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gIgbUK2fBQGW7ekPtBlXAL4um1NhJ9HA4ZfsnksaiAs=; b=GoVgA2vzN28H9rlPdQg7pRmRQ702JpqUuzS9+JdE78tBFxuc7pvsVtYMWdPzeT5Zjw OIzXYpqJ+PtrA/oJBP8U4YUrWhAID2rXAAw4csdRIwmHVAcY0V+Ub/ohmdYe9hB79zMG 3QK6JwVClFpw/nBAps3itHeiuG9Rjo+OLrFCH9OQNSTY6o7/5+WKqvRfGxTCY+ofnhDR 5glLyeMKt1SLNpApGik3hnBOH0zcRSErw+NQNXZYd5xqNHZDY0YC9HBRuDvT6uWRr5b1 yhGEgaRopLC0MAC/1KU1MJPa1m10dYYx1TdpGf+lbXozuXL0swo9qF0oKN4BlyHI0eqk YgIA== X-Gm-Message-State: APt69E0TyMQ52u/Rpv9C6DVe9qFPIGboKLYeX0WP0QsZlOBr2p2Fju1P mwlWiNmNQ5RII0Knsbl6vcpYRw== X-Google-Smtp-Source: ADUXVKJK3VWEvftFWKAuGCvvmjsnYKGK+CI5b5A3/5qeqESBt9zws4TOUjeKMv1Hk9NOhUsllwAf2Q== X-Received: by 2002:a1c:3504:: with SMTP id c4-v6mr1628433wma.59.1529500836388; Wed, 20 Jun 2018 06:20:36 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: cota@braap.org, famz@redhat.com, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Wed, 20 Jun 2018 14:20:30 +0100 Message-Id: <20180620132032.12952-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180620132032.12952-1-alex.bennee@linaro.org> References: <20180620132032.12952-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22f Subject: [Qemu-devel] [RFC PATCH 3/5] .travis.yml: add gcovr summary for GCOV build X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 This gives a more useful summary, sorted by descending % coverage, after the tests have run. The final numbers will give an idea if our coverage is getting better or worse. As quite a lot of lines don't get covered at all we filter out all the 0% lines. If the file doesn't appear it is not being exercised. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- .travis.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.travis.yml b/.travis.yml index fabfe9ec34..83e0577464 100644 --- a/.travis.yml +++ b/.travis.yml @@ -38,6 +38,7 @@ addons: - libvte-2.90-dev - sparse - uuid-dev + - gcovr =20 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu # to prevent IRC notifications from forks. This was created using: @@ -81,6 +82,8 @@ matrix: compiler: clang # gprof/gcov are GCC features - env: CONFIG=3D"--enable-gprof --enable-gcov --disable-pie --target-l= ist=3Daarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,= ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu" + after_success: + - gcovr -p | grep -v "0%" | sed s/[0-9]\*[,-]//g compiler: gcc # We manually include builds which we disable "make check" for - env: CONFIG=3D"--enable-debug --enable-tcg-interpreter" --=20 2.17.1 From nobody Sat May 4 23:46:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529501255810678.6660343173355; Wed, 20 Jun 2018 06:27:35 -0700 (PDT) Received: from localhost ([::1]:49673 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd95-0005LO-1o for importer@patchew.org; Wed, 20 Jun 2018 09:27:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54996) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd2N-0007yD-Ds for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVd2M-0002lJ-88 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:39 -0400 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]:40168) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVd2M-0002kq-22 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:38 -0400 Received: by mail-wm0-x244.google.com with SMTP id n5-v6so6850999wmc.5 for ; Wed, 20 Jun 2018 06:20:37 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id d3-v6sm3430119wri.24.2018.06.20.06.20.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 06:20:34 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 7E01E3E08E5; Wed, 20 Jun 2018 14:20:33 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OVAp7QMxQs2WEQyoZm93Y02LjcKymtDnW4R825/T0iQ=; b=bxdnKNtm0ttbSyP/oUG+k+EnoYPTJrEj2Rv2/OvGIvTxBckCMGrzpxzdtCSWjgV3DK G+ilxJ4OW8vzJoFQ1Nv3HAr1uOvwOJl+xuuJL8PygXsqHpsaFn6R/D9hKUfRsbUsiJbf 5jxICekqfnUPIc+55u+9LMRB4OejaxjzUNyrs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OVAp7QMxQs2WEQyoZm93Y02LjcKymtDnW4R825/T0iQ=; b=OWU+C7yjDEjqYvvzJ06J21oZhydUPkdy5rMnEAFFFnTOpUkNwn0WH4HpPWjZNpSPFb xfuEKcmQltGCBLaTtsjRe7uyf1L7daAnABgXAnaLRGhnuJGDDckvIHV2RmIBH5uV3Ryh R/lbo2vjWCNj6DLyaZFY255IPdTV6TTMRvWczq/YNK5O1Rvd7ZkUbxGMFcJzDdWg2M/y VXmu3lYPMpEZLvmh9NcfthhzsnK79ZiHQrJUhPbZnI3w4c+rTvuO8kHFeJ1j4hx3ZxvN xbssqC2WG66Etu1hH+x0BU2zNQV022kpm7v7TK+r93UBL4wkqMdist232F8iGfmYBIAG ewOQ== X-Gm-Message-State: APt69E2Hf8GwS/RHcBKMTmHxzeSjCviO62mnZaPrM9JIVYTcZ0ZqmPZn fZfH8vfR0q2g3DOztcdlQg3Ftg== X-Google-Smtp-Source: ADUXVKL+YttySFX29C1KJV5fbYLbKQEwutJXLRAdBCC/SqpY3X3hrhT88qbXisxq3unr01XXVTLkcQ== X-Received: by 2002:a1c:e846:: with SMTP id f67-v6mr1706778wmh.63.1529500837096; Wed, 20 Jun 2018 06:20:37 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: cota@braap.org, famz@redhat.com, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Wed, 20 Jun 2018 14:20:31 +0100 Message-Id: <20180620132032.12952-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180620132032.12952-1-alex.bennee@linaro.org> References: <20180620132032.12952-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::244 Subject: [Qemu-devel] [RFC PATCH 4/5] build-system: add clean-coverage target X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 This can be used to remove any stale coverage data before any particular test run. This is useful for analysing individual tests. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- Makefile | 11 +++++++++++ docs/devel/testing.rst | 11 ++++++++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index e46f2b625a..cb4af8bf80 100644 --- a/Makefile +++ b/Makefile @@ -725,6 +725,14 @@ module_block.h: $(SRC_PATH)/scripts/modules/module_blo= ck.py config-host.mak $(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \ "GEN","$@") =20 +ifdef CONFIG_GCOV +.PHONY: clean-coverage +clean-coverage: + $(call quiet-command, \ + find . \( -name '*.gcda' -o -name '*.gcov' \) -type f -exec rm {} +, \ + "CLEAN", "coverage files") +endif + clean: # avoid old build problems by removing potentially incorrect old files rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h ge= n-op-arm.h @@ -1075,6 +1083,9 @@ endif echo '') @echo 'Cleaning targets:' @echo ' clean - Remove most generated files but keep the conf= ig' +ifdef CONFIG_GCOV + @echo ' clean-coverage - Remove coverage files' +endif @echo ' distclean - Remove all generated files' @echo ' dist - Build a distributable tarball' @echo '' diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 66ef219f69..a3652aea14 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -161,9 +161,14 @@ GCC gcov support ``gcov`` is a GCC tool to analyze the testing coverage by instrumenting the tested code. To use it, configure QEMU with ``--enable-gcov`` option and build. Then run ``make check`` as usual. -Reports can be obtained by running ``gcov`` command on the output -files under ``$build_dir/tests/``, please read the ``gcov`` -documentation for more information. + +If you want to gather coverage information on a single test the ``make +clean-coverage`` target can be used to any existing coverage +information before running a single test. + +Reports can be obtained by running ``gcov`` command +on the output files under ``$build_dir/tests/``, please read the +``gcov`` documentation for more information. =20 QEMU iotests =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.17.1 From nobody Sat May 4 23:46:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1529501415300338.9665358683586; Wed, 20 Jun 2018 06:30:15 -0700 (PDT) Received: from localhost ([::1]:49688 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVdBe-0008Sp-KH for importer@patchew.org; Wed, 20 Jun 2018 09:30:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55063) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVd2P-000809-A2 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVd2N-0002mI-GY for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:41 -0400 Received: from mail-wr0-x244.google.com ([2a00:1450:400c:c0c::244]:34551) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVd2N-0002ld-4w for qemu-devel@nongnu.org; Wed, 20 Jun 2018 09:20:39 -0400 Received: by mail-wr0-x244.google.com with SMTP id a12-v6so3306000wro.1 for ; Wed, 20 Jun 2018 06:20:39 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id w24-v6sm1970475wmc.15.2018.06.20.06.20.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 06:20:37 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id A4A8C3E08E8; Wed, 20 Jun 2018 14:20:33 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pAwjhM+QKaOn/D5Q7G6m80xibAAi7GmM7LxpKJMtf9Y=; b=Ai3D1Rghi7dRZNwQNJAwN+tgrdqVV6anoE17mDbc9UFETb476D5LdAE0QXwA9k93J+ brIcnOd0oBj/eoVeWbrB0mOKn9L4gHPAQzs4NrX7kNPp+/gevnU6n5lpUtAwC5Q5zkyI Pm6AYa7v3x4sNM84JNWPDd/k3xPYRR7e5l1og= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pAwjhM+QKaOn/D5Q7G6m80xibAAi7GmM7LxpKJMtf9Y=; b=Mc00fWYqXPAZsuHquot0Tt1pv4rD5NXoh5e68CLIr+3N/WlWiQynivMw7j1M02rRIW 7zbCSYqOMJJbp+p19FVNa4h5DyyfkLk0630kk4959Eu9K+njeDgQab9oOxeKVzvmq4S/ RtY92rvjcLde0jk4v4SBZrhwgvienxhhtacxUvGrkpOiLJBxwAROXmniugu3pWEeADKu DWH3qtgVYDpm0OcBWGxBAeds6kT0XfDLyI+8Cmlg5lRzGVvns1UxPqjElLu0bumZKcsm 8vg43aTvMgaZobawhvBpPiSZOf31LmbfyN+WJMuMiwh7ifvpb1jteclnPMBv34tqq8Nv LMvQ== X-Gm-Message-State: APt69E1sjtZES83bXcJhczgwfFNXFfX1tmUlfMCwPr/QBgGXcCkDsS/p p6IFpABFAX1CBDxDf06fKFgtxg== X-Google-Smtp-Source: ADUXVKKD2cU06oxNumv/t3ebassbb4aVvupYMZBPWRNG/MoPQnIuymXWVwlwtQhjeZTcC0Xqeme2UQ== X-Received: by 2002:adf:a35b:: with SMTP id d27-v6mr1763804wrb.189.1529500838179; Wed, 20 Jun 2018 06:20:38 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: cota@braap.org, famz@redhat.com, berrange@redhat.com, f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com, aurelien@aurel32.net, agraf@suse.de Date: Wed, 20 Jun 2018 14:20:32 +0100 Message-Id: <20180620132032.12952-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180620132032.12952-1-alex.bennee@linaro.org> References: <20180620132032.12952-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::244 Subject: [Qemu-devel] [RFC PATCH 5/5] build-system: add coverage-report target X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 This will build a coverage report under the current directory in reports/coverage. At the users option a report can be generated by directly invoking something like: make foo/bar/coverage-report.html Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- Makefile | 13 +++++++++++++ docs/devel/testing.rst | 11 ++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index cb4af8bf80..7450e0b7b5 100644 --- a/Makefile +++ b/Makefile @@ -988,6 +988,16 @@ docs/interop/qemu-qmp-ref.dvi docs/interop/qemu-qmp-re= f.html \ docs/interop/qemu-qmp-ref.txt docs/interop/qemu-qmp-ref.7: \ docs/interop/qemu-qmp-ref.texi docs/interop/qemu-qmp-qapi.texi =20 +# Reports/Analysis + +%/coverage-report.html: + @mkdir -p $* + $(call quiet-command,\ + gcovr -p --html --html-details -o $@, \ + "GEN", "coverage-report.html") + +.PHONY: coverage-report +coverage-report: $(CURDIR)/reports/coverage/coverage-report.html =20 ifdef CONFIG_WIN32 =20 @@ -1097,6 +1107,9 @@ endif @echo 'Documentation targets:' @echo ' html info pdf txt' @echo ' - Build documentation in specified format' +ifdef CONFIG_GCOV + @echo ' coverage-report - Create code coverage report' +endif @echo '' ifdef CONFIG_WIN32 @echo 'Windows targets:' diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index a3652aea14..9dcdd19260 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -166,9 +166,14 @@ If you want to gather coverage information on a single= test the ``make clean-coverage`` target can be used to any existing coverage information before running a single test. =20 -Reports can be obtained by running ``gcov`` command -on the output files under ``$build_dir/tests/``, please read the -``gcov`` documentation for more information. +You can generate a HTML coverage report by executing ``make +coverage-report`` which will generate into +./reports/coverage/coverage-report.html. If you want to generate it +elsewhere simply execute ``make /foo/bar/baz/coverage-report.html``. + +Further analysis can be conducted by running the ``gcov`` command +directly on the various .gcda output files. Please read the ``gcov`` +documentation for more information. =20 QEMU iotests =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.17.1