From nobody Thu Sep 19 16:30:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1722965634; cv=none; d=zohomail.com; s=zohoarc; b=NhRuGh1q0fBxfPvBduppiYA5GQXNWeFX2qXe3vIsbPi7j1toE7/lm2oRf2j8KBMIYMjE3T7kntg5jVCvm1x+hSJQdzA+OWXeaC/6nn2RZuM9+cruemYSkFO4J21iIojPmMtHAqenrV+uzppyN+qHWVquNUMusZ8OWFxhAFe1ipU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1722965634; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nf0S6+2UWAoiK8QJhCNo+DMvi4LJgeUAgWrNmgWlrjc=; b=Zc4DQi3gxWLXMwsOBg+XGwdpfIQ2O+KPCRKlRSn0HWDPubcP/TY0dkksk9R27hikwrSB7WfQf/e0I6Q7ZfWUl4nHGuDGh7+XjFebvvScPv3YWipIdkIOoJg4RFG6lS0LInPNBULL/vPCoie81IATZwv/uSQl1HiMk48z4VOHRpw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1722965634435297.0480323235689; Tue, 6 Aug 2024 10:33:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sbO3v-0004MV-Gr; Tue, 06 Aug 2024 13:33:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sbO3s-0003xt-W2 for qemu-devel@nongnu.org; Tue, 06 Aug 2024 13:33:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sbO3r-0006Vu-B3 for qemu-devel@nongnu.org; Tue, 06 Aug 2024 13:33:28 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-z6V6pmDHM0m4MjOMBls8sg-1; Tue, 06 Aug 2024 13:33:22 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A773F1955D42; Tue, 6 Aug 2024 17:33:15 +0000 (UTC) Received: from p1.localdomain.com (unknown [10.39.192.15]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 809E819560AE; Tue, 6 Aug 2024 17:33:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1722965606; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nf0S6+2UWAoiK8QJhCNo+DMvi4LJgeUAgWrNmgWlrjc=; b=TkhlzPS74T1RjmKNXAL+vUU3QmxQDvhqalWlI1+i3g+cB1Q9uvaKfcq9sCmZM/7wo7LE4j WfjJevnl0yECsioNB4FWABcGygHkM/IezBXwPcUkIs36TOSIGw91gYozXHuPw0NDl2T4B0 aMACGhctTsshVh2MgckEsMO3TtNiTDg= X-MC-Unique: z6V6pmDHM0m4MjOMBls8sg-1 From: Cleber Rosa To: qemu-devel@nongnu.org Cc: Marcin Juszkiewicz , Pavel Dovgalyuk , Radoslaw Biernacki , Troy Lee , Akihiko Odaki , Beraldo Leal , kvm@vger.kernel.org, Joel Stanley , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Aurelien Jarno , Sriram Yagnaraman , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Paul Durrant , Eric Auger , David Woodhouse , qemu-arm@nongnu.org, Cleber Rosa , Andrew Jeffery , Jamin Lin , Steven Lee , Peter Maydell , Yoshinori Sato , Wainer dos Santos Moschetta , Thomas Huth , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Leif Lindholm Subject: [PATCH v2 9/9] Avocado tests: allow for parallel execution of tests Date: Tue, 6 Aug 2024 13:31:19 -0400 Message-ID: <20240806173119.582857-10-crosa@redhat.com> In-Reply-To: <20240806173119.582857-1-crosa@redhat.com> References: <20240806173119.582857-1-crosa@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.144, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1722965634914116600 Content-Type: text/plain; charset="utf-8" The updated Avocado version allows for the execution of tests in parallel. While on a CI environment it may not be a good idea to increase the parallelization level in a single runner, developers may leverage that on specific CI runners or on their development environments. This also multiplies the timeout for each test accordingly. The reason is that more concurrency can lead to less resources, and less resources can lead to some specific tests taking longer to complete and then time out. The timeout factor being used here is very conservative (being equal to the amount of parallel tasks). The worst this possibly oversized timeout value can do is making users wait a bit longer for the job to finish if a test hangs. Overall, users can expect a much quicker turnaround on most systems with a value such as 8 on a 12 core machine. Signed-off-by: Cleber Rosa --- docs/devel/testing.rst | 12 ++++++++++++ tests/Makefile.include | 6 +++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index af73d3d64f..97ebc8211f 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -998,6 +998,18 @@ of Avocado or ``make check-avocado``, and can also be = queried using: =20 pyvenv/bin/avocado list tests/avocado =20 +To run tests in parallel, the ``AVOCADO_PARALLEL`` environment +variable can be defined with a value different than ``1`` (its default +value). Example: + + .. code:: + + make check-avocado AVOCADO_PARALLEL=3D4 + +Please exercise care when using parallel execution with the QEMU +Avocado tests as a higher system load can cause time sensitive tests +to timeout and be interrupted. + Manual Installation ~~~~~~~~~~~~~~~~~~~ =20 diff --git a/tests/Makefile.include b/tests/Makefile.include index 537804d101..545b5155f9 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -94,6 +94,9 @@ TESTS_RESULTS_DIR=3D$(BUILD_DIR)/tests/results ifndef AVOCADO_TESTS AVOCADO_TESTS=3Dtests/avocado endif +ifndef AVOCADO_PARALLEL + AVOCADO_PARALLEL=3D1 +endif # Controls the output generated by Avocado when running tests. # Any number of command separated loggers are accepted. For more # information please refer to "avocado --help". @@ -141,7 +144,8 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-i= mages --show=3D$(AVOCADO_SHOW) run --job-results-dir=3D$(TESTS_RESUL= TS_DIR) \ $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ --filter-by-tags-include-empty-key) \ - $(AVOCADO_CMDLINE_TAGS) --max-parallel-tasks=3D1 \ + $(AVOCADO_CMDLINE_TAGS) --max-parallel-tasks=3D$(AVOCADO_PARAL= LEL) \ + -p timeout_factor=3D$(AVOCADO_PARALLEL) \ $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \ "AVOCADO", "tests/avocado") =20 --=20 2.45.2