From nobody Sat Nov 15 00:43:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1758200347; cv=none; d=zohomail.com; s=zohoarc; b=d6suaX7K9vioz69F5EfCTg67XuZH+OTN/mM2OuN2NC+phnw3Pndq1cNxuQPPHnL19bfD8JF1hV3vFdEevsAUb9A0u3syIQ6+yVf5imX3cg12pqSE2tZDJy0PnPnzGL74lW5oB1K1EH6ceY28wDCEMgMlUA1wwr1SNu4iFo4g/sY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1758200347; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QzXClJ6DHGH/mQMLYudLFDnvzvpbXhbMBy6spScgMEk=; b=Ekff1mk+lhxpqQ4SYeJtIsumGvno4Uwy4NEovkBKGdw6IARoEse8o+bCqcjwIArYwf3ad8QxrL5uX2gohPDNRur/1ZggY0MyRZHRNWufXkyE8RHG0tbFIHnDR4HAqCsOe1npMxt/ZNZLUQbbEnUYPwkcrNadLw4ghHB1bqgMHqQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1758200347058209.01378367866914; Thu, 18 Sep 2025 05:59:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uzEDG-0003Wr-6T; Thu, 18 Sep 2025 08:58:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uzED7-0003IS-5n for qemu-devel@nongnu.org; Thu, 18 Sep 2025 08:58:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uzECz-00053U-Ub for qemu-devel@nongnu.org; Thu, 18 Sep 2025 08:58:04 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-196-tqW4lwojMvedFwsqJ13n9g-1; Thu, 18 Sep 2025 08:57:53 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 385F319560BB; Thu, 18 Sep 2025 12:57:52 +0000 (UTC) Received: from toolbx.redhat.com (unknown [10.42.28.161]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A85991955F19; Thu, 18 Sep 2025 12:57:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1758200275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QzXClJ6DHGH/mQMLYudLFDnvzvpbXhbMBy6spScgMEk=; b=YfNSWeb4n87/KzeQ6l0wfMWFdZ5XLI7W2sTf7QtgSHbOCm/rnNsGcCnOwdBay7TPhcCuI2 NtLgQKjOKicvhmSfY1EHCo/3Hbk+bzY0CjhUtNPXogb3tys5MrAPu8qZky6fCHCMXGpmob tYpmGl/vnOi/7uWxaTRr+cuSRN8gyIc= X-MC-Unique: tqW4lwojMvedFwsqJ13n9g-1 X-Mimecast-MFC-AGG-ID: tqW4lwojMvedFwsqJ13n9g_1758200272 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH 1/2] tests/functional: retry when seeing ConnectionError exception Date: Thu, 18 Sep 2025 13:57:45 +0100 Message-ID: <20250918125746.1165658-2-berrange@redhat.com> In-Reply-To: <20250918125746.1165658-1-berrange@redhat.com> References: <20250918125746.1165658-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.005, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1758200348753116601 This base class is used for many different socket connection errors, corresponding to ECONNRESET, ECONNREFUSED, ECONNABORTED and more. Most of these are things you might expect to see every now and then as transient flaws. We should thus retry the asset download when seeing them. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Thomas Huth --- tests/functional/qemu_test/asset.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_te= st/asset.py index 2dd32bf28d..f8b87d2153 100644 --- a/tests/functional/qemu_test/asset.py +++ b/tests/functional/qemu_test/asset.py @@ -179,6 +179,13 @@ def fetch(self): self.url, e.reason) raise AssetError(self, "Unable to download: URL error %s" % e.reason, transient=3DTrue) + except ConnectionError as e: + # A socket connection failure, such as dropped conn + # or refused conn + tmp_cache_file.unlink() + self.log.error("Unable to download %s: Connection error %s= ", + self.url, e) + continue except Exception as e: tmp_cache_file.unlink() raise AssetError(self, "Unable to download: %s" % e) --=20 2.50.1 From nobody Sat Nov 15 00:43:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1758200347; cv=none; d=zohomail.com; s=zohoarc; b=MZXXdcwIYOEsCk2BLD0ms6TlqAlmu5UHW+ZfHEirQRx382DTuNPcVtvN2NKJ9QVSRwEGMNfBxXqBSMrWKMTLIzJDcp3Xsst5wjbiR9KtOfOkkhczDEkyMow0v/ufVrOXUyWHJmlzQMLwUyV7sFlCG4hijIhP9ImEVTk1jbjXfVU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1758200347; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=cBV5ap63WOrCrneSt7ZG9xFTyaARxzUk2hdnQoVzOik=; b=hMBMbaBf6dkMAfwHiqDg5DfnmmFNTQhO7epJGrFwYUC6zeI4+5xQ9/kiQwFwep8tQOcjJM/ozmDTr+Ms+f7TXYwACfRZFh+G320+T+6j2+rluJ5HqeFL8O+lYnAydEgkZwmPsX/1TNIK0Onot6pVND2OlFF8Ehe7GDaqaxDmJJ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175820034788773.81560317722722; Thu, 18 Sep 2025 05:59:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uzEDE-0003VD-Hd; Thu, 18 Sep 2025 08:58:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uzED7-0003If-87 for qemu-devel@nongnu.org; Thu, 18 Sep 2025 08:58:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uzED5-00056W-NQ for qemu-devel@nongnu.org; Thu, 18 Sep 2025 08:58:05 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-591-G9j3y099M5e8aglrJk6xPQ-1; Thu, 18 Sep 2025 08:57:55 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 53E4A180028C; Thu, 18 Sep 2025 12:57:54 +0000 (UTC) Received: from toolbx.redhat.com (unknown [10.42.28.161]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E07DF1955F21; Thu, 18 Sep 2025 12:57:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1758200279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cBV5ap63WOrCrneSt7ZG9xFTyaARxzUk2hdnQoVzOik=; b=MqhDMCn+kycAfBzYTcqhoyZKr5CnvGQ3eFV6KS++yBs1QoBpzEkekLu1GM3owdpqmXzZPZ pMF7ghMWwNNnkFv1zS3yypPhg5Gf5wvfStQAUwiM+/QesjyPxEDGxe7l413owJAor9TkUe e5+yDwAnRkKWuKF9v25tgNX/oQZBOTc= X-MC-Unique: G9j3y099M5e8aglrJk6xPQ-1 X-Mimecast-MFC-AGG-ID: G9j3y099M5e8aglrJk6xPQ_1758200274 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH 2/2] tests/functional: treat unknown exceptions as transient faults Date: Thu, 18 Sep 2025 13:57:46 +0100 Message-ID: <20250918125746.1165658-3-berrange@redhat.com> In-Reply-To: <20250918125746.1165658-1-berrange@redhat.com> References: <20250918125746.1165658-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.005, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1758200348767116600 To maximise the robustness of the functional tests we want to treat most asset download failures as non-fatal to the test suite. Instead it should just skip the tests which need that particular asset. The only time aim to make it fatal is for 404 errors which are highly likely to reflect genuine problems to be fixed. We catch certain exception classes and handle them as transient errors, but unfortunately it is proving difficult to predict what exception classes urlopen() is capable of raising, with new possibilities being discovered. To provide a fail-safe, treat the generic Exception class as being a transient error too. This may well mask certain genuine bugs, but it is preferrable to prioritize running the test suite to the greatest extent practical. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Thomas Huth --- tests/functional/qemu_test/asset.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_te= st/asset.py index f8b87d2153..2971a989d1 100644 --- a/tests/functional/qemu_test/asset.py +++ b/tests/functional/qemu_test/asset.py @@ -188,7 +188,8 @@ def fetch(self): continue except Exception as e: tmp_cache_file.unlink() - raise AssetError(self, "Unable to download: %s" % e) + raise AssetError(self, "Unable to download: %s" % e, + transient=3DTrue) =20 if not os.path.exists(tmp_cache_file): raise AssetError(self, "Download retries exceeded", transient= =3DTrue) --=20 2.50.1