From nobody Wed Nov 27 02:38:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1701443683; cv=none; d=zohomail.com; s=zohoarc; b=YrLirhpkz43WVFaHAbd4dIXc2QmG+3a3adWGCmyUoUDjyPGclczNchcksb5W0fCFKJQPh7evLdUhn4JPD9QYLjznffhx2a9ryAmrJWqnnPBgPllgHsl65ntLZBA+kevUOmCtbJROf1piHOIJnxdI6xV+nwjh8MEG2IL0Mq7eTQc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1701443683; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=57x23omuVrExXgPDfxrIdHt1R1VqyRD4qvLTNT8D6tI=; b=b6W00mQWlxMuDgUGiIrV1+N2f5DavDPGR7GWfv2K7yI+zjFghpf+NVtGlMWkNrssiEfJEI2hUXCZm6QHETjy1d2YXYL2PYg7DkbtIaaV6sc6fEh3iAY0V4w3htg6CggdDpjSNmflLc9h9NOSDZt+VZVD+SMaRNAH8/GgP44Iwdo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1701443683155832.4497151850567; Fri, 1 Dec 2023 07:14:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r95Cq-000583-3H; Fri, 01 Dec 2023 10:13:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r95CX-00052l-HA for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:13:09 -0500 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r95CS-00053c-3m for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:13:09 -0500 Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-40b538d5c4eso17076035e9.1 for ; Fri, 01 Dec 2023 07:13:00 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id v10-20020adfe4ca000000b00332f95ab451sm4390896wrm.58.2023.12.01.07.12.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 07:12:56 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 00BB45FB61; Fri, 1 Dec 2023 15:12:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701443580; x=1702048380; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=57x23omuVrExXgPDfxrIdHt1R1VqyRD4qvLTNT8D6tI=; b=eIKZVJhUzGBzmdGVEfWRhAZe+jQQWaxG5pHOSFyZBZsRCqmD2hmxF/P96rprv+H+T2 FBrlNtKQAfnMg6Il9UlW4V+kUHjdK63RrpYrPpjColwcyKqSJ16ie0H1OQYCyIYSy+IX JE3UbMYxhkxawEd14J/nSfWlxKjalrdyRe2g0wvVGqWQX9kVldW1pEUX2cWu/OS5fLsY ut0C7VEbpbNfKXTmqsdcZi0Sby65PD0RdtLs97ie2DwoilL/1FJYZZbZI6JSgg+r5bwX ZJjkS2H9RiC87Lm09mOu6B1/QU54guumXrEeUH+MSVJtAiEdyqxEgwpfnO2TsfLcePNs iv6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701443580; x=1702048380; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=57x23omuVrExXgPDfxrIdHt1R1VqyRD4qvLTNT8D6tI=; b=Xg2wNJbcV/1x1N8Pfg1m8VWXOfuue7XCKPAFA7/kg9LQOtZZwvL4JMz7R+m/u0DiOe YajROpmfEUUISacZ3Yk6zTAmWsvx99tBAtEBGznzjKJHNXBCelCvEtG6adWfqp3Q0neB SczBuGG3xePzlbflY0IeiZcRBYVUXKOuXuDm9FUJPi6ZbJTupddRlrWNDKYoXInAfc1Z E2shA3OeRcruDa7gi+I0WFKvrGPBCwovFTCHwvkzc9O59nuaQN2toOeRXCTmdy4uYjaZ by/k/nHUx31L4hxVk16ch94N8FecyLT7eecwG6SBkRICVC3n8uVUnJneHHVzenQlP/pA BRIA== X-Gm-Message-State: AOJu0YxliCxYdmEQ4N9ldC1YESuYNdIy5FV956kqncmUKtIvvlfxPQxI FSCYA+carQJRETZEgbVJ+2HCfg== X-Google-Smtp-Source: AGHT+IGG6uiBxj1FeOOcjK4MvU5HBQ+rVuiZEPQlv41LIbyZGclfq0hWmbiVzBK5Hrkssi5OBAEJ6Q== X-Received: by 2002:adf:f48d:0:b0:333:2fd2:4add with SMTP id l13-20020adff48d000000b003332fd24addmr896006wro.89.1701443579792; Fri, 01 Dec 2023 07:12:59 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Luis Machado , Richard Henderson , Stefan Hajnoczi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 1/6] gdbstub: use a better signal when we halt for IO reasons Date: Fri, 1 Dec 2023 15:12:23 +0000 Message-Id: <20231201151228.2610209-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231201151228.2610209-1-alex.bennee@linaro.org> References: <20231201151228.2610209-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::334; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1701443683606000001 The gdb description GDB_SIGNAL_IO is "I/O possible" and by default gdb will try and restart the guest, getting us nowhere. Report GDB_SIGNAL_STOP instead which should at least halt the session at the failure point. Reviewed-by: Luis Machado Reviewed-by: Richard Henderson Acked-by: Stefan Hajnoczi Signed-off-by: Alex Benn=C3=A9e Message-Id: <20231201093633.2551497-2-alex.bennee@linaro.org> diff --git a/gdbstub/internals.h b/gdbstub/internals.h index 465c24b36e..5c0c725e54 100644 --- a/gdbstub/internals.h +++ b/gdbstub/internals.h @@ -24,6 +24,7 @@ enum { GDB_SIGNAL_TRAP =3D 5, GDB_SIGNAL_ABRT =3D 6, GDB_SIGNAL_ALRM =3D 14, + GDB_SIGNAL_STOP =3D 17, GDB_SIGNAL_IO =3D 23, GDB_SIGNAL_XCPU =3D 24, GDB_SIGNAL_UNKNOWN =3D 143 diff --git a/gdbstub/system.c b/gdbstub/system.c index 783ac140b9..83fd452800 100644 --- a/gdbstub/system.c +++ b/gdbstub/system.c @@ -183,7 +183,7 @@ static void gdb_vm_state_change(void *opaque, bool runn= ing, RunState state) break; case RUN_STATE_IO_ERROR: trace_gdbstub_hit_io_error(); - ret =3D GDB_SIGNAL_IO; + ret =3D GDB_SIGNAL_STOP; break; case RUN_STATE_WATCHDOG: trace_gdbstub_hit_watchdog(); --=20 2.39.2 From nobody Wed Nov 27 02:38:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1701443673; cv=none; d=zohomail.com; s=zohoarc; b=OakK267H5oTBFQfTNo5TThimwVgmQvKlUjJ7EAnLzxC9oetpncluyeKA1IlJ1FT8OLSsjObqWV3lSMcMcIVr8+9dGlSxTW7wMO9aTfxgjLynojVXN9jtW1l2rDv6ZcLsN8pZ9gHIOMtPq7wani4NsDT3p0Or3urFavhdkoynjoo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1701443673; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CILzwn6E0PdXeCb4qj1dxi6nAuVedehhKSy9oJRmnFs=; b=i+TUlvXA0I2E2gnM7Dmfqgx0wIu6mLdgqefRbcHzd5p4byYPaBm4QlWAnPvffoepgw1Ye6525cpzWJKLLo/Fxsz8aMNpcivDI9nzuGd7AHsW8Kn6YFVmCLjn4jJwDPOIN/EkfnPtIki4a59dBMinZmZUhFEvQDE5WEl+K3zUUVY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1701443673520826.1002053057848; Fri, 1 Dec 2023 07:14:33 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r95CW-00050L-4T; Fri, 01 Dec 2023 10:13:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r95C1-0004rA-Gm for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:42 -0500 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r95Bv-0003ss-GO for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:37 -0500 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-40b4f60064eso22260025e9.1 for ; Fri, 01 Dec 2023 07:12:31 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id a13-20020a05600c348d00b0040b5377cf03sm9420700wmq.1.2023.12.01.07.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 07:12:29 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 218245FB63; Fri, 1 Dec 2023 15:12:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701443550; x=1702048350; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CILzwn6E0PdXeCb4qj1dxi6nAuVedehhKSy9oJRmnFs=; b=iR1ax6BhxVk5A5LGGuIFfntG9UKlZjzJU6vlxbSNtPpcG0Wa6lgYew7EuNZx8zLgGR ft0KoIuZc7ZD+FV+LiUff7eh8Z+cUttQCnUJHjS3X9pqRj5ZvyXyvaivcrNN5No9F7a7 GXCnyJOLcNujdEs3uGv0KlwiMaLNE9n+LXU2YvIjp1cmm+8MCPi3a1So7F6VLsx6+7cV vl6a4u64B/oWNTJn0LgyLaSwfZ3hLNBfxx2vQdYa2OW8X2wMxXzVEJRZg0CPz1mKhaJP iOgseyjFYJW6FWRwgDIh4f4y43n9JJ1O8G/RfQZT5psojQII5Dxndeh3iVlHgCk4bl5f nVAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701443550; x=1702048350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CILzwn6E0PdXeCb4qj1dxi6nAuVedehhKSy9oJRmnFs=; b=a6+YSvW/IgAwDKzGsVgTIgBISohutr8JTi1MykhOsGloc46BvP/Cs8HUIN7D+hJn7+ m0YFCWoCkyHZa+j7DXiQNU96uWlJ44H19VS9lxpJWYBKDzR4q3ylJ5Q8yisSyPEH4fJI zc0P3ivcXSo03Py+tkUPT7My2/oH1F5sCs/NXsQKuy5p7Ynd2NvaKrQyQmoS4lqeCMKs O2dSUFPlDEUzcS/kCa62InjsQAQQUzZnleMhgkPD0Sfc4sI/QqiURLgZkjXYv7CPus2R x3NpiqPVBRzcWTHko0ZC5FZo+QoVVAcgs+fS2wZCL6wcFxkwfQoFUfmKNVvvj4dl0c3V meWQ== X-Gm-Message-State: AOJu0YzpNvp5wHdfhqFxXihwUlD624NEPmBJ3gyajHehnbEFDp4GPGWV 8n5MeNTKtk1idXZU06fTSOMDHQ== X-Google-Smtp-Source: AGHT+IG8xY1Q4RXBKn2JJQRswoW42sjM6PolpjAZpEHBwZGOTpeOaxwm4a3uIG8Oe0SYwUnH9GjWkA== X-Received: by 2002:a7b:c45a:0:b0:40b:5f03:b42e with SMTP id l26-20020a7bc45a000000b0040b5f03b42emr219917wmi.336.1701443550024; Fri, 01 Dec 2023 07:12:30 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson , Stefan Hajnoczi , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal , Cleber Rosa , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Maydell , Andrew Jeffery , Joel Stanley , Aurelien Jarno , Yoshinori Sato , Halil Pasic , Christian Borntraeger , Eric Farman , Pavel Dovgalyuk , Paolo Bonzini , Eric Auger , qemu-arm@nongnu.org (open list:ASPEED BMCs), qemu-s390x@nongnu.org (open list:S390 Virtio-ccw) Subject: [PULL 2/6] docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS Date: Fri, 1 Dec 2023 15:12:24 +0000 Message-Id: <20231201151228.2610209-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231201151228.2610209-1-alex.bennee@linaro.org> References: <20231201151228.2610209-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x332.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1701443677314000003 It doesn't make sense to have two classes of flaky tests. While it may take the constrained environment of CI to trigger failures easily it doesn't mean they don't occasionally happen on developer machines. As CI is the gating factor to passing there is no point developers running the tests locally anyway unless they are trying to fix things. While we are at it update the language in the docs to discourage the QEMU_TEST_FLAKY_TESTS becoming a permanent solution. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Richard Henderson Acked-by: Stefan Hajnoczi Signed-off-by: Alex Benn=C3=A9e Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 22218dbedb..76465b8f3d 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -1371,23 +1371,32 @@ conditions. For example, tests that take longer to = execute when QEMU is compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` var= iable has been used to determine whether those tests should run or not. =20 -GITLAB_CI -^^^^^^^^^ -A number of tests are flagged to not run on the GitLab CI. Usually because -they proved to the flaky or there are constraints on the CI environment wh= ich -would make them fail. If you encounter a similar situation then use that -variable as shown on the code snippet below to skip the test: +QEMU_TEST_FLAKY_TESTS +^^^^^^^^^^^^^^^^^^^^^ +Some tests are not working reliably and thus are disabled by default. +This includes tests that don't run reliably on GitLab's CI which +usually expose real issues that are rarely seen on developer machines +due to the constraints of the CI environment. If you encounter a +similar situation then raise a bug and then mark the test as shown on +the code snippet below: =20 .. code:: =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on Git= Lab') def test(self): do_something() =20 -QEMU_TEST_FLAKY_TESTS -^^^^^^^^^^^^^^^^^^^^^ -Some tests are not working reliably and thus are disabled by default. -Set this environment variable to enable them. +Tests should not live in this state forever and should either be fixed +or eventually removed. + +To run such tests locally you will need to set the environment +variable. For example: + +.. code:: + + env QEMU_TEST_FLAKY_TESTS=3D1 ./pyvenv/bin/avocado run \ + tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg =20 Uninstalling Avocado ~~~~~~~~~~~~~~~~~~~~ diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py index be30dcbd58..9e9773e6e1 100644 --- a/tests/avocado/boot_linux.py +++ b/tests/avocado/boot_linux.py @@ -12,7 +12,7 @@ =20 from avocado_qemu import LinuxTest, BUILD_DIR =20 -from avocado import skipIf +from avocado import skipUnless =20 =20 class BootLinuxX8664(LinuxTest): @@ -93,7 +93,8 @@ class BootLinuxPPC64(LinuxTest): =20 timeout =3D 360 =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_pseries_tcg(self): """ :avocado: tags=3Dmachine:pseries @@ -111,7 +112,8 @@ class BootLinuxS390X(LinuxTest): =20 timeout =3D 240 =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_s390_ccw_virtio_tcg(self): """ :avocado: tags=3Dmachine:s390-ccw-virtio diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux= _console.py index 6eab515718..231b4f68e5 100644 --- a/tests/avocado/boot_linux_console.py +++ b/tests/avocado/boot_linux_console.py @@ -15,7 +15,7 @@ =20 from avocado import skip from avocado import skipUnless -from avocado import skipIf +from avocado import skipUnless from avocado_qemu import QemuSystemTest from avocado_qemu import exec_command from avocado_qemu import exec_command_and_wait_for_pattern @@ -1419,7 +1419,8 @@ def test_ppc_mac99(self): # This test has a 6-10% failure rate on various hosts that look # like issues with a buggy kernel. As a result we don't want it # gating releases on Gitlab. - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_sh4_r2d(self): """ :avocado: tags=3Darch:sh4 diff --git a/tests/avocado/intel_iommu.py b/tests/avocado/intel_iommu.py index 77635ab56c..2dd11a6346 100644 --- a/tests/avocado/intel_iommu.py +++ b/tests/avocado/intel_iommu.py @@ -9,10 +9,11 @@ # later. See the COPYING file in the top-level directory. import os =20 -from avocado import skipIf +from avocado import skipUnless from avocado_qemu import LinuxTest =20 -@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') +@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLa= b') + class IntelIOMMU(LinuxTest): """ :avocado: tags=3Darch:x86_64 diff --git a/tests/avocado/linux_initrd.py b/tests/avocado/linux_initrd.py index ba02e5a563..c40a987bd1 100644 --- a/tests/avocado/linux_initrd.py +++ b/tests/avocado/linux_initrd.py @@ -13,7 +13,7 @@ import tempfile =20 from avocado_qemu import QemuSystemTest -from avocado import skipIf +from avocado import skipUnless =20 =20 class LinuxInitrd(QemuSystemTest): @@ -53,7 +53,8 @@ def test_with_2gib_file_should_exit_error_msg_with_linux_= v3_6(self): max_size + 1) self.assertRegex(self.vm.get_log(), expected_msg) =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_with_2gib_file_should_work_with_linux_v4_16(self): """ QEMU has supported up to 4 GiB initrd for recent kernel diff --git a/tests/avocado/machine_aspeed.py b/tests/avocado/machine_aspeed= .py index df31b2a8a8..258fb50c47 100644 --- a/tests/avocado/machine_aspeed.py +++ b/tests/avocado/machine_aspeed.py @@ -18,7 +18,7 @@ from avocado_qemu import interrupt_interactive_console_until_pattern from avocado_qemu import has_cmd from avocado.utils import archive -from avocado import skipIf +from avocado import skipUnless from avocado import skipUnless =20 =20 @@ -311,7 +311,8 @@ def do_test_arm_aspeed_sdk_start(self, image): self, 'boot', '## Loading kernel from FIT Image') self.wait_for_console_pattern('Starting kernel ...') =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_arm_ast2500_evb_sdk(self): """ :avocado: tags=3Darch:arm @@ -329,7 +330,8 @@ def test_arm_ast2500_evb_sdk(self): self.workdir + '/ast2500-default/image-bmc') self.wait_for_console_pattern('nodistro.0 ast2500-default ttyS4') =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_arm_ast2600_evb_sdk(self): """ :avocado: tags=3Darch:arm diff --git a/tests/avocado/machine_mips_malta.py b/tests/avocado/machine_mi= ps_malta.py index 9bd54518bf..959dcf5602 100644 --- a/tests/avocado/machine_mips_malta.py +++ b/tests/avocado/machine_mips_malta.py @@ -11,7 +11,7 @@ import gzip import logging =20 -from avocado import skipIf +from avocado import skipUnless from avocado import skipUnless from avocado.utils import archive from avocado_qemu import QemuSystemTest @@ -101,7 +101,8 @@ def test_mips_malta_i6400_framebuffer_logo_1core(self): """ self.do_test_i6400_framebuffer_logo(1) =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_mips_malta_i6400_framebuffer_logo_7cores(self): """ :avocado: tags=3Darch:mips64el @@ -111,7 +112,8 @@ def test_mips_malta_i6400_framebuffer_logo_7cores(self): """ self.do_test_i6400_framebuffer_logo(7) =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_mips_malta_i6400_framebuffer_logo_8cores(self): """ :avocado: tags=3Darch:mips64el diff --git a/tests/avocado/machine_rx_gdbsim.py b/tests/avocado/machine_rx_= gdbsim.py index 6cd8704b01..350a73fbbb 100644 --- a/tests/avocado/machine_rx_gdbsim.py +++ b/tests/avocado/machine_rx_gdbsim.py @@ -10,7 +10,7 @@ =20 import os =20 -from avocado import skipIf +from avocado import skipUnless from avocado_qemu import QemuSystemTest from avocado_qemu import exec_command_and_wait_for_pattern from avocado_qemu import wait_for_console_pattern @@ -22,7 +22,8 @@ class RxGdbSimMachine(QemuSystemTest): timeout =3D 30 KERNEL_COMMON_COMMAND_LINE =3D 'printk.time=3D0 ' =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_uboot(self): """ U-Boot and checks that the console is operational. @@ -46,7 +47,8 @@ def test_uboot(self): # FIXME limit baudrate on chardev, else we type too fast #exec_command_and_wait_for_pattern(self, 'version', gcc_version) =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_linux_sash(self): """ Boots a Linux kernel and checks that the console is operational. diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/avocado/machi= ne_s390_ccw_virtio.py index ffd914ded9..61e75d8f9e 100644 --- a/tests/avocado/machine_s390_ccw_virtio.py +++ b/tests/avocado/machine_s390_ccw_virtio.py @@ -12,7 +12,7 @@ import os import tempfile =20 -from avocado import skipIf +from avocado import skipUnless from avocado_qemu import QemuSystemTest from avocado_qemu import exec_command_and_wait_for_pattern from avocado_qemu import wait_for_console_pattern diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py index 53cb7e5091..0d32cc280e 100644 --- a/tests/avocado/replay_kernel.py +++ b/tests/avocado/replay_kernel.py @@ -15,7 +15,7 @@ import time =20 from avocado import skip -from avocado import skipIf +from avocado import skipUnless from avocado import skipUnless from avocado_qemu import wait_for_console_pattern from avocado.utils import archive @@ -82,6 +82,7 @@ def run_rr(self, kernel_path, kernel_command_line, consol= e_pattern, =20 class ReplayKernelNormal(ReplayKernelBase): =20 + # See https://gitlab.com/qemu-project/qemu/-/issues/2010 @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets s= tuck') def test_x86_64_pc(self): """ @@ -179,7 +180,8 @@ def test_arm_virt(self): =20 self.run_rr(kernel_path, kernel_command_line, console_pattern, shi= ft=3D1) =20 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_arm_cubieboard_initrd(self): """ :avocado: tags=3Darch:arm diff --git a/tests/avocado/reverse_debugging.py b/tests/avocado/reverse_deb= ugging.py index ed04e92bb4..9a468321e5 100644 --- a/tests/avocado/reverse_debugging.py +++ b/tests/avocado/reverse_debugging.py @@ -10,7 +10,7 @@ import os import logging =20 -from avocado import skipIf +from avocado import skipUnless from avocado_qemu import BUILD_DIR from avocado.utils import datadrainer from avocado.utils import gdb @@ -206,7 +206,8 @@ def get_pc(self, g): + self.get_reg_le(g, self.REG_CS) * 0x10 =20 # unidentified gitlab timeout problem - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_x86_64_pc(self): """ :avocado: tags=3Darch:x86_64 @@ -223,7 +224,8 @@ class ReverseDebugging_AArch64(ReverseDebugging): REG_PC =3D 32 =20 # unidentified gitlab timeout problem - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_aarch64_virt(self): """ :avocado: tags=3Darch:aarch64 @@ -247,7 +249,8 @@ class ReverseDebugging_ppc64(ReverseDebugging): REG_PC =3D 0x40 =20 # unidentified gitlab timeout problem - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_ppc64_pseries(self): """ :avocado: tags=3Darch:ppc64 @@ -260,7 +263,8 @@ def test_ppc64_pseries(self): self.reverse_debugging() =20 # See https://gitlab.com/qemu-project/qemu/-/issues/1992 - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + def test_ppc64_powernv(self): """ :avocado: tags=3Darch:ppc64 diff --git a/tests/avocado/smmu.py b/tests/avocado/smmu.py index b3c4de6bf4..05b34418a5 100644 --- a/tests/avocado/smmu.py +++ b/tests/avocado/smmu.py @@ -9,10 +9,11 @@ # later. See the COPYING file in the top-level directory. import os =20 -from avocado import skipIf +from avocado import skipUnless from avocado_qemu import LinuxTest, BUILD_DIR =20 -@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') +@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLa= b') + class SMMU(LinuxTest): """ :avocado: tags=3Daccel:kvm diff --git a/tests/avocado/tuxrun_baselines.py b/tests/avocado/tuxrun_basel= ines.py index c99bea6c0b..5f859f4e6f 100644 --- a/tests/avocado/tuxrun_baselines.py +++ b/tests/avocado/tuxrun_baselines.py @@ -13,7 +13,7 @@ import time import tempfile =20 -from avocado import skip, skipIf +from avocado import skip, skipUnless from avocado_qemu import QemuSystemTest from avocado_qemu import exec_command, exec_command_and_wait_for_pattern from avocado_qemu import wait_for_console_pattern @@ -551,7 +551,7 @@ def test_s390(self): haltmsg=3D"Requesting system halt") =20 # Note: some segfaults caused by unaligned userspace access - @skipIf(os.getenv('GITLAB_CI'), 'Skipping unstable test on GitLab') + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') def test_sh4(self): """ :avocado: tags=3Darch:sh4 --=20 2.39.2 From nobody Wed Nov 27 02:38:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1701443665; cv=none; d=zohomail.com; s=zohoarc; b=SmflLKZ8zCE3+LJik/gyMyU7c4eUQneV32JHibbgjIrv938HX0hZZd4lkV29YUot0y3CErmkOZJboQHRCEgcZy5d3marwVBPxuAjxkM4Ly/nQ8Xw6l/SJjdXQLQRSkN8/7+HTnzdzqp2HRXJ5NrHMiAhBSQFf3ANwa8Jds6P0tk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1701443665; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kxGGSJ63Zm4VVOM52878zzwTuRazNRD8JON/droSv9Q=; b=XTbVUV/KI25fD23aCbxD/sl2D0QfjzE+p1DEpmU1+KWeHNeW3xcxOnPyi08nvsbc8hr611weXU8ValRpmngbcs0tY3sgZMrNLgEJX3W7nmu3ARu3aWC33VK8pATi8/YjV5SXugBJbNH7/m4NyOp90jx38w8MNnbK1G6DJ8xGEaI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 170144366548541.86445393495603; Fri, 1 Dec 2023 07:14:25 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r95CW-00050R-6Y; Fri, 01 Dec 2023 10:13:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r95Bz-0004qj-7n for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:36 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r95Bv-0003so-GD for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:34 -0500 Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-40838915cecso21272695e9.2 for ; Fri, 01 Dec 2023 07:12:31 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id t3-20020adfe443000000b00332fcdafc22sm4466794wrm.70.2023.12.01.07.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 07:12:29 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 33A5A5FB65; Fri, 1 Dec 2023 15:12:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701443550; x=1702048350; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kxGGSJ63Zm4VVOM52878zzwTuRazNRD8JON/droSv9Q=; b=uIkYsq9y9CCiTgWBsobFrWlsdq0TfKDz3GMb+fj9hlG+Yfpu8mmcE9/TKhFWwCMVlD kyHQ70D+GrEHQe5VRr76SACk664AvQp9mSjwtZdhB/n01aO7s4nvirl+GRVupYcoHues ytaZjhdqlfNiZcYAHCSQawIUmWswvi7NVBGz4h4dBvdQSunnkUMr6KNGCO/fqozeZSQV 1eeSb2JIq4oTfb+MfYbdL3Enz+Zwq5jcSaNk7qBTTggwZloLyHbJq2YpixmmVe6qrovX Xn3YS8zNOneX91I7y/285iQthW//rRqt9o4bOVG4GyjdIIFh6xlk68YfWUREsZarmehn XG5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701443550; x=1702048350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kxGGSJ63Zm4VVOM52878zzwTuRazNRD8JON/droSv9Q=; b=iXuZ5TqhgLdUkMuHTPwsMGKc10lt4S9VqrqNdkU1WPHyTZjxzjldPa3VivAy9Swsrf xDgMu0OjfVNB51GckdIdMZhAy1CTagJa5M89KH6VPL2Bon8vV0ks08Air02OrltikrRR PTjbqH+vpWIDyRFWWRi6xPoY8KVAR+bGgo4rXWSo4+bNXDQitE+ius1CVsYGR02RospH 2lnKy4o9x3CtyaayBATmeYk8MAsZUdLYLM8uaCxbi5ekVgk7ooYaQoc/DYZn44LbWoB5 0a+STEBsDWi9BoCgzCG7W5nBicE1a3ABz8QioBMgfOTx/SEHFJjdBgFFkaHKZQtKp+bu u1Vg== X-Gm-Message-State: AOJu0YyLAYLR+bn+sqfZr3AMBTFQItMfDMOMYgeXvtg4+CbKA5vedxQ5 7j9hExc3Wm6UMD8pHczl9B7qpA== X-Google-Smtp-Source: AGHT+IEz+ed1qN3UiQ8+6xcESqLyfNeUWW8mMYrB+Z1SPk14j8FOh5qPy1RHOo1CeeOP4eY+RSwv2w== X-Received: by 2002:a05:600c:4296:b0:40b:5e59:daa9 with SMTP id v22-20020a05600c429600b0040b5e59daa9mr453832wmc.188.1701443549806; Fri, 01 Dec 2023 07:12:29 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Stefan Hajnoczi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Cleber Rosa , Wainer dos Santos Moschetta , Beraldo Leal , Jiaxun Yang Subject: [PULL 3/6] tests/avocado: drop BootLinuxConsole.test_mips_malta_cpio test Date: Fri, 1 Dec 2023 15:12:25 +0000 Message-Id: <20231201151228.2610209-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231201151228.2610209-1-alex.bennee@linaro.org> References: <20231201151228.2610209-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32f; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1701443667503000001 The assets are no longer archived by Debian so we can't run this on CI. While some people may still have the test in their cache we do have more recent images from tuxrun so this isn't a great loss. Cc: Stefan Hajnoczi Cc: Philippe Mathieu-Daud=C3=A9 Acked-by: Stefan Hajnoczi Signed-off-by: Alex Benn=C3=A9e Message-Id: <20231201093633.2551497-4-alex.bennee@linaro.org> diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux= _console.py index 231b4f68e5..ba2f99b53a 100644 --- a/tests/avocado/boot_linux_console.py +++ b/tests/avocado/boot_linux_console.py @@ -191,47 +191,6 @@ def test_mips64el_fuloong2e(self): console_pattern =3D 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) =20 - def test_mips_malta_cpio(self): - """ - :avocado: tags=3Darch:mips - :avocado: tags=3Dmachine:malta - :avocado: tags=3Dendian:big - """ - deb_url =3D ('http://snapshot.debian.org/archive/debian/' - '20160601T041800Z/pool/main/l/linux/' - 'linux-image-4.5.0-2-4kc-malta_4.5.5-1_mips.deb') - deb_hash =3D 'a3c84f3e88b54e06107d65a410d1d1e8e0f340f8' - deb_path =3D self.fetch_asset(deb_url, asset_hash=3Ddeb_hash) - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinux-4.5.0-2-4kc-mal= ta') - initrd_url =3D ('https://github.com/groeck/linux-build-test/raw/' - '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/' - 'mips/rootfs.cpio.gz') - initrd_hash =3D 'bf806e17009360a866bf537f6de66590de349a99' - initrd_path_gz =3D self.fetch_asset(initrd_url, asset_hash=3Dinitr= d_hash) - initrd_path =3D self.workdir + "rootfs.cpio" - archive.gzip_uncompress(initrd_path_gz, initrd_path) - - self.vm.set_console() - kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE - + 'console=3DttyS0 console=3Dtty ' - + 'rdinit=3D/sbin/init noreboot') - self.vm.add_args('-kernel', kernel_path, - '-initrd', initrd_path, - '-append', kernel_command_line, - '-no-reboot') - self.vm.launch() - self.wait_for_console_pattern('Boot successful.') - - exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo', - 'BogoMIPS') - exec_command_and_wait_for_pattern(self, 'uname -a', - 'Debian') - exec_command_and_wait_for_pattern(self, 'reboot', - 'reboot: Restarting system= ') - # Wait for VM to shut down gracefully - self.vm.wait() - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code= ') def test_mips64el_malta_5KEc_cpio(self): """ --=20 2.39.2 From nobody Wed Nov 27 02:38:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1701443620; cv=none; d=zohomail.com; s=zohoarc; b=Aw54LfF208SL1MZQy8DmlHXUxA57BoEAEGY4PB7R3KiaNppGvh0GWwZiCOh4lnQ8V2zs0jczFx1901AAofbTBpkLWWhhchE+txnAtXyebT8ruFr3ZN1w2TkhmAZHpZA4LOEmWsiBXSFWJjC0DKm5A9su0mQPua5ORGU0P/tw6DA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1701443620; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LTrDNbYJOBcXJe0QIsla+RGqqidagZ/wagIa9X/ca1U=; b=R3qbEPVHM9KtXuPttsm8FoCDKTIa1ME5TaLntFSro9HYkxTXrAjWe4OpXWeg3QPGdAVf3oDVckbHS2ICXEJIBjZ/BnGHZbVN+bY6hZm/duithmCJkxWJGTuscPGeibS+kEwnfgWzonktHi+bD2Vzy31YLzLYGrh5KbBSxY3Tf4g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1701443620780525.107947305766; Fri, 1 Dec 2023 07:13:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r95CG-0004t6-Ug; Fri, 01 Dec 2023 10:12:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r95C0-0004qm-Lt for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:36 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r95By-0003tD-Kq for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:36 -0500 Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-40b595bf5d2so18221265e9.2 for ; Fri, 01 Dec 2023 07:12:32 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id p8-20020adfcc88000000b0032f9688ea48sm4432922wrj.10.2023.12.01.07.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 07:12:29 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 453C65FB67; Fri, 1 Dec 2023 15:12:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701443551; x=1702048351; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LTrDNbYJOBcXJe0QIsla+RGqqidagZ/wagIa9X/ca1U=; b=lQ0UGDdsVNJG2WGR7RUw95wjbmZfd9A3ORuVSilG/27w62yrkpEKOrKUBEkIPYxIae DaXav+08hmjhdwccp8RgonWA/qqBEUeZf5j5vz/qvz/iaCTKkNYgSBV22RNooEdAe/xq fnhxsBjKXrRuGYAjJHtJRuVxW38X7xwC9HjIiqSQFe9kEgSOX3yDH2i/5Zfcnyc0PU1E 0HERjNNFEntYO1sp7Uzfbbl15ZLkQe/y9WmwRfKM9D5XI3kmGkrTqjaZ48ufKcnGQJS9 3+x7+3OJO2IJubpfid2ySVuFU7AZ3lvOmUmKF3+oD+TMY5N3nJ+ntqFdpufj5lkTPS3h Ubxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701443551; x=1702048351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LTrDNbYJOBcXJe0QIsla+RGqqidagZ/wagIa9X/ca1U=; b=G/VnmTyIKhqfGb90c3kBnoDSZQthiZ+S8lX84rG+yLQKaXUsiRXQ16Yv61a8fnkrRR FiGEOa04wCEelqE1Wp19nRnEm3Y+1zMncKJVWo6ACuO1pBFHpnryK6V15bCw4P8oYP5d /HY1OJG6P19VRGOdlG5mzgB/PZdGNSQ7MLhMgZomX0PrnrnMpuF09Vm5/cbmxNqCMoFI I22+v2ne4Rv4g4iiXUiMI5Q1P1Ic3kY8/ydVkTA9jKf0355sjqg5TlG2D4QYfKlNTiH6 N9ACsmG50DTMIzxN/YNxqzm9a2sjhBR4YntaI+XxnZFI9rr36F5g6ild5Z1kgXVZ7uyn Mpng== X-Gm-Message-State: AOJu0YxWe7CM4sVM4IB/uZzUQVwIRgjH+2YXhs4NsA4dKXyVa9IhyDrA E1mdDeLJ1csK5WfIH8LujW0azg== X-Google-Smtp-Source: AGHT+IFIXGsAnlt/Y2TNLGMkoYJVTHmE6Bb5mgXSldoqSHN+PXzG6tO17uyxxO6NLD8iHdCTOmLBqA== X-Received: by 2002:a5d:540b:0:b0:333:2fd2:3c0c with SMTP id g11-20020a5d540b000000b003332fd23c0cmr648167wrv.197.1701443551790; Fri, 01 Dec 2023 07:12:31 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Radoslaw Biernacki , Peter Maydell , Leif Lindholm , Marcin Juszkiewicz , Cleber Rosa , Wainer dos Santos Moschetta , Beraldo Leal , qemu-arm@nongnu.org (open list:SBSA-REF) Subject: [PULL 4/6] tests/avocado: tag sbsa tests as tcg only Date: Fri, 1 Dec 2023 15:12:26 +0000 Message-Id: <20231201151228.2610209-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231201151228.2610209-1-alex.bennee@linaro.org> References: <20231201151228.2610209-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32f; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1701443621707000001 As firmware runs at a higher privilege level than the hypervisor we can only run these tests under TCG emulation. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Acked-by: Stefan Hajnoczi Signed-off-by: Alex Benn=C3=A9e Message-Id: <20231201093633.2551497-6-alex.bennee@linaro.org> diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machi= ne_aarch64_sbsaref.py index bdd1efc768..528c7d2934 100644 --- a/tests/avocado/machine_aarch64_sbsaref.py +++ b/tests/avocado/machine_aarch64_sbsaref.py @@ -20,6 +20,10 @@ class Aarch64SbsarefMachine(QemuSystemTest): """ :avocado: tags=3Darch:aarch64 :avocado: tags=3Dmachine:sbsa-ref + :avocado: tags=3Daccel:tcg + + As firmware runs at a higher privilege level than the hypervisor we + can only run these tests under TCG emulation. """ =20 timeout =3D 180 --=20 2.39.2 From nobody Wed Nov 27 02:38:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1701443668; cv=none; d=zohomail.com; s=zohoarc; b=Ne+WnHHgPPsVBHs9G00+zwAy9IGc6T7NHe8d6ez4xXmFe/I+yuaUPFn5iZHx9Ap2zoOETOGn7ZPEr/uuf7k4AnrCS7N8hXxZxzW1OAofPRZrF9hRAk52OQz/3fvnALLqqpN/QobZdiDxNpf124VSKKj9yIOiHq45+cyVoAlpTuQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1701443668; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=x+4RIhvIWUcpCT7qUh814AJlZGLCqs4LOMeRimhH8r0=; b=gzLnqo25gG1MRQXP/hR9rIVtzgV0BKMCgKbnqYbCnLKVrBGsRimy5CSZo8FlUN0Fl1plMmSqp7N+o3QGYXd1VxQ3zmHas6TTSBfxy/FFuscF/tnfvzdpSmkZF7+1wSXz1dZXuQaDmXc8Oijq6Qb01Lqcr/+gZnTJZIfsGlBuE24= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1701443668822903.9244049828629; Fri, 1 Dec 2023 07:14:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r95Cc-00053w-AG; Fri, 01 Dec 2023 10:13:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r95C4-0004rF-50 for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:42 -0500 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r95Bz-00040A-DH for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:39 -0500 Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-40b40423df8so19504115e9.0 for ; Fri, 01 Dec 2023 07:12:34 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id bg36-20020a05600c3ca400b003fe1fe56202sm5800283wmb.33.2023.12.01.07.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 07:12:31 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 568205FB69; Fri, 1 Dec 2023 15:12:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701443554; x=1702048354; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x+4RIhvIWUcpCT7qUh814AJlZGLCqs4LOMeRimhH8r0=; b=fg2X59czmtDQxdrGIcVWNwFufWDX6/u5b0S7ERubPODP7cyt1YlbhlFPYGyPCkYieM FlhaqHaERwF98g0GVLDiTQclWzCAZoPqR6HpefsK8wMeqkHNow1EkT6ezLlT8GnCcrZw zo0mz6Rd10cvUYOCzfepmfbFvuU0teb0TjTrwGgDh3j5JHFuUnLpIKHbaiU4KQ+TYH5o d1QXzeXHud39iWcqnKLFDhFCK7n2i6xnhMQwv1C0Rwd9sutLY055LheLX0TcTSC0G1dr 2Jk+7NeynbWtxUqbZe80U3kMRlkHm7ZZS6RAWdnjFPSDbofy3ifaW2MBHypnDRL6lLP5 /BcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701443554; x=1702048354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x+4RIhvIWUcpCT7qUh814AJlZGLCqs4LOMeRimhH8r0=; b=Q/xwIQ0AH2uxo1MG2+wdHKmRVp132jRlGKBRTQv4SNzBTRCnxfLihnDKV50v5ti/Fq /BEXLvPA9p0ZB+c5ff66BlAspNB8kGedisEdJIcYkeeSfT9DyL3lsdMIPHRlNxxhmQLo Z7E4AxzROh5PNLNbWI0I6JikuncfaksCT6+k3qSrnnEMqb/bkdx/0SLotHX3ea7aswUL caEDKjr63/iQ1HNQlp3pKzxXA7SqpBZ+H4MLbwhfBAJ9sdlCuCqdihDsZJDGjcC38y7t +xBclfA9akPl5XLFDzOrIEOnzUm1c4P7jDp++M8sRK4iM6ozyoULrDG5AAgUl2gVFvE+ Rg+w== X-Gm-Message-State: AOJu0Yw8eCqky4vTw9iTxHl3mzXyUk2kRZACsHXPGkT0rySR6Y50x6wH YL0DdTuyi9Z0NiEJSQ0lXzbKcg== X-Google-Smtp-Source: AGHT+IH3EwFR/B2ezm6hg3FPVhAUZVDSRWwVPahZhRORlaZSAYWfKLdhcYGgjVsvuiD5HkXMYex+pQ== X-Received: by 2002:a05:600c:2317:b0:40b:5e4a:409a with SMTP id 23-20020a05600c231700b0040b5e4a409amr347344wmo.186.1701443553654; Fri, 01 Dec 2023 07:12:33 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Stefan Hajnoczi , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal Subject: [PULL 5/6] gitlab: build the correct microblaze target Date: Fri, 1 Dec 2023 15:12:27 +0000 Message-Id: <20231201151228.2610209-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231201151228.2610209-1-alex.bennee@linaro.org> References: <20231201151228.2610209-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1701443669521000005 We inadvertently built the LE target for BE tests. Fixes: 78ebc00b06 (gitlab: shuffle some targets and reduce avocado noise) Reviewed-by: Philippe Mathieu-Daud=C3=A9 Acked-by: Stefan Hajnoczi Signed-off-by: Alex Benn=C3=A9e Message-Id: <20231201093633.2551497-7-alex.bennee@linaro.org> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 7f9af83b10..62b5379a5e 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -41,7 +41,7 @@ build-system-ubuntu: variables: IMAGE: ubuntu2204 CONFIGURE_ARGS: --enable-docs - TARGETS: alpha-softmmu microblazeel-softmmu mips64el-softmmu + TARGETS: alpha-softmmu microblaze-softmmu mips64el-softmmu MAKE_CHECK_ARGS: check-build =20 check-system-ubuntu: --=20 2.39.2 From nobody Wed Nov 27 02:38:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1701443692; cv=none; d=zohomail.com; s=zohoarc; b=GCY/Trs/H5Rd+s8K2u9udkytAZPhtcKnNHlzjZ/LLzdTChTsl01iclvEUXh45dZMndTAsnDmsUZKa6RpLF0BdEM+2QSkUniGms6k9Hw8yJEquPsBh0B2bTjsBta41dQi+HTFCQCP7Qic/Y0EyygkVw6doPhje29gf21N9QitQQU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1701443692; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5LRig+tQmW4bH55Vbco9n1/RCIy3/FKKNzzRuQiv8bw=; b=I2PgwbmbsvW01EAV/sutihJPWUjDr29c00IEYojNN3f+jNJUL0u9NZVv3RfVf5qrcBRi3AeWtw/KE3d1G6+hBidTf4WNebKAscTKvXZcr0KGAENGjqZCFAJrgUCKuoUempWJup4cL9U+xSh9Z5VDwMyuqoQjBP7Bs9bpuWYXl/0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1701443692669461.32698737482906; Fri, 1 Dec 2023 07:14:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r95CY-000532-6J; Fri, 01 Dec 2023 10:13:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r95C4-0004rE-36 for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:42 -0500 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r95By-0003tc-Uo for qemu-devel@nongnu.org; Fri, 01 Dec 2023 10:12:39 -0500 Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-40a4848c6e1so19382795e9.1 for ; Fri, 01 Dec 2023 07:12:33 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id j7-20020a5d4527000000b00332fda15056sm4387336wra.110.2023.12.01.07.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 07:12:31 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 782DF5FB6B; Fri, 1 Dec 2023 15:12:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1701443552; x=1702048352; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5LRig+tQmW4bH55Vbco9n1/RCIy3/FKKNzzRuQiv8bw=; b=yeOsBj9jqQQnk0Vn7/s1MOPC/2JGV4/VZn7dq4YUlRLNURJP7ua3GSqbINEz7/CrZk 8pQhfkjKS6jGM+WAOISQYj2qxZEjFL3Q614HQT12XjfhwiB5bPGzRXnemFBkhr1+t+e2 kTq8yzcT+9PnmBt6zzVIVy4r9DE6OJ3GHZfjVLrXRwgLa0SsfJzoQOwNgj8KE89Np0gy /J1eKwC3U15UTUbi4r6WE9Yq4UGEzrVjcmgFUa4nVKViBBlYF92VyM80Eebdf5LLvTBh x8bjHnLlD2u7yvb4y2m8QSXb8Kg5qUzq8CL0u2aQ0DDTu0h9YGDkIe1UYipC/1ZnG6Uy x1UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701443552; x=1702048352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5LRig+tQmW4bH55Vbco9n1/RCIy3/FKKNzzRuQiv8bw=; b=U1ZBh6OUm+ycq27kglortlHSTKQ2et1o5oamrqMzmU52qs7z2IGBfVsvFWMutblYMb 5OmmT8QKnw1yNMM5mRWqpXnqB9NUBc1ff2RDM0seAOhCTs2P+gd7gHZdzc5/yWFBrRXL J8gDFcSZHlOvXTqogW+U3Ufb7c4iptGLFjUIBa7OT4nvu6FtapXyvoJyFo2F6IKeQ9qs KuY7Jyo3tbC+qYgCEma9P/pP0J3U8Ibx4mPBrAZrkMuFJ/KH7Js+rpgI2ySfW9/StCQo AhzJ5c1ouyya4jypBHo7xkmMXhWX3pAAqIUUsQW0PThKXZWU8ia1R8NBngRqBBvak0CO kjjA== X-Gm-Message-State: AOJu0YwvPeZ0KKGbIjDxl/sUzgkqSiz+J+jAwwo9k7jBtdF1DWznRdHs alif3aJVbEy+yxjNFiamQCHs4w== X-Google-Smtp-Source: AGHT+IEQTUVX3zI65OB2ry2Aks32KoHnSROybFXEWTTAea9jbLb6pbszkHVB5vb01vhfTVEg5abeyw== X-Received: by 2002:a05:600c:4f46:b0:40b:5e4a:4068 with SMTP id m6-20020a05600c4f4600b0040b5e4a4068mr376375wmq.136.1701443552133; Fri, 01 Dec 2023 07:12:32 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Richard Henderson , Stefan Hajnoczi , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal , Cleber Rosa , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Peter Maydell , Andrew Jeffery , Joel Stanley , Aurelien Jarno , Yoshinori Sato , Halil Pasic , Christian Borntraeger , Eric Farman , Pavel Dovgalyuk , Paolo Bonzini , Eric Auger , qemu-arm@nongnu.org (open list:ASPEED BMCs), qemu-s390x@nongnu.org (open list:S390 Virtio-ccw) Subject: [PULL 6/6] gitlab: add optional job to run flaky avocado tests Date: Fri, 1 Dec 2023 15:12:28 +0000 Message-Id: <20231201151228.2610209-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231201151228.2610209-1-alex.bennee@linaro.org> References: <20231201151228.2610209-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1701443693910000003 One problem with flaky tests is they often only fail under CI conditions which makes it hard to debug. We add an optional allow_fail job so developers can trigger the only the flaky tests in the CI environment if they are debugging. Reviewed-by: Richard Henderson Acked-by: Stefan Hajnoczi Signed-off-by: Alex Benn=C3=A9e Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 76465b8f3d..bd132306c1 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -1387,16 +1387,17 @@ the code snippet below: def test(self): do_something() =20 -Tests should not live in this state forever and should either be fixed -or eventually removed. - -To run such tests locally you will need to set the environment -variable. For example: +You can also add ``:avocado: tags=3Dflaky`` to the test meta-data so +only the flaky tests can be run as a group: =20 .. code:: =20 - env QEMU_TEST_FLAKY_TESTS=3D1 ./pyvenv/bin/avocado run \ - tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg + env QEMU_TEST_FLAKY_TESTS=3D1 ./pyvenv/bin/avocado \ + run tests/avocado -filter-by-tags=3Dflaky + +Tests should not live in this state forever and should either be fixed +or eventually removed. + =20 Uninstalling Avocado ~~~~~~~~~~~~~~~~~~~~ diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 62b5379a5e..91663946de 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -217,6 +217,36 @@ avocado-system-opensuse: MAKE_CHECK_ARGS: check-avocado AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64 =20 +# +# Flaky tests. We don't run these by default and they are allow fail +# but often the CI system is the only way to trigger the failures. +# + +build-system-flaky: + extends: + - .native_build_job_template + - .native_build_artifact_template + needs: + job: amd64-debian-container + variables: + IMAGE: debian + QEMU_JOB_OPTIONAL: 1 + TARGETS: aarch64-softmmu arm-softmmu mips64el-softmmu + ppc64-softmmu rx-softmmu s390x-softmmu sh4-softmmu x86_64-softmmu + MAKE_CHECK_ARGS: check-build + +avocado-system-flaky: + extends: .avocado_test_job_template + needs: + - job: build-system-flaky + artifacts: true + allow_failure: true + variables: + IMAGE: debian + MAKE_CHECK_ARGS: check-avocado + QEMU_JOB_OPTIONAL: 1 + QEMU_TEST_FLAKY_TESTS: 1 + AVOCADO_TAGS: flaky =20 # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by # the configure script. The container doesn't contain Xen headers so diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py index 9e9773e6e1..7c4769904e 100644 --- a/tests/avocado/boot_linux.py +++ b/tests/avocado/boot_linux.py @@ -99,6 +99,7 @@ def test_pseries_tcg(self): """ :avocado: tags=3Dmachine:pseries :avocado: tags=3Daccel:tcg + :avocado: tags=3Dflaky """ self.require_accelerator("tcg") self.vm.add_args("-accel", "tcg") @@ -118,6 +119,7 @@ def test_s390_ccw_virtio_tcg(self): """ :avocado: tags=3Dmachine:s390-ccw-virtio :avocado: tags=3Daccel:tcg + :avocado: tags=3Dflaky """ self.require_accelerator("tcg") self.vm.add_args("-accel", "tcg") diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux= _console.py index ba2f99b53a..7487337f46 100644 --- a/tests/avocado/boot_linux_console.py +++ b/tests/avocado/boot_linux_console.py @@ -1384,6 +1384,7 @@ def test_sh4_r2d(self): """ :avocado: tags=3Darch:sh4 :avocado: tags=3Dmachine:r2d + :avocado: tags=3Dflaky """ tar_hash =3D 'fe06a4fd8ccbf2e27928d64472939d47829d4c7e' self.vm.add_args('-append', 'console=3DttySC1') diff --git a/tests/avocado/intel_iommu.py b/tests/avocado/intel_iommu.py index 2dd11a6346..f04ee1cf9d 100644 --- a/tests/avocado/intel_iommu.py +++ b/tests/avocado/intel_iommu.py @@ -22,6 +22,7 @@ class IntelIOMMU(LinuxTest): :avocado: tags=3Dmachine:q35 :avocado: tags=3Daccel:kvm :avocado: tags=3Dintel_iommu + :avocado: tags=3Dflaky """ =20 IOMMU_ADDON =3D ',iommu_platform=3Don,disable-modern=3Doff,disable-leg= acy=3Don' diff --git a/tests/avocado/linux_initrd.py b/tests/avocado/linux_initrd.py index c40a987bd1..aad5b19bd9 100644 --- a/tests/avocado/linux_initrd.py +++ b/tests/avocado/linux_initrd.py @@ -57,6 +57,8 @@ def test_with_2gib_file_should_exit_error_msg_with_linux_= v3_6(self): =20 def test_with_2gib_file_should_work_with_linux_v4_16(self): """ + :avocado: tags=3Dflaky + QEMU has supported up to 4 GiB initrd for recent kernel Expect guest can reach 'Unpacking initramfs...' """ diff --git a/tests/avocado/machine_aspeed.py b/tests/avocado/machine_aspeed= .py index 258fb50c47..6fa5459a07 100644 --- a/tests/avocado/machine_aspeed.py +++ b/tests/avocado/machine_aspeed.py @@ -317,6 +317,7 @@ def test_arm_ast2500_evb_sdk(self): """ :avocado: tags=3Darch:arm :avocado: tags=3Dmachine:ast2500-evb + :avocado: tags=3Dflaky """ =20 image_url =3D ('https://github.com/AspeedTech-BMC/openbmc/releases= /' @@ -336,6 +337,7 @@ def test_arm_ast2600_evb_sdk(self): """ :avocado: tags=3Darch:arm :avocado: tags=3Dmachine:ast2600-evb + :avocado: tags=3Dflaky """ =20 image_url =3D ('https://github.com/AspeedTech-BMC/openbmc/releases= /' diff --git a/tests/avocado/machine_mips_malta.py b/tests/avocado/machine_mi= ps_malta.py index 959dcf5602..99bee49e9a 100644 --- a/tests/avocado/machine_mips_malta.py +++ b/tests/avocado/machine_mips_malta.py @@ -109,6 +109,7 @@ def test_mips_malta_i6400_framebuffer_logo_7cores(self): :avocado: tags=3Dmachine:malta :avocado: tags=3Dcpu:I6400 :avocado: tags=3Dmips:smp + :avocado: tags=3Dflaky """ self.do_test_i6400_framebuffer_logo(7) =20 @@ -120,6 +121,7 @@ def test_mips_malta_i6400_framebuffer_logo_8cores(self): :avocado: tags=3Dmachine:malta :avocado: tags=3Dcpu:I6400 :avocado: tags=3Dmips:smp + :avocado: tags=3Dflaky """ self.do_test_i6400_framebuffer_logo(8) =20 diff --git a/tests/avocado/machine_rx_gdbsim.py b/tests/avocado/machine_rx_= gdbsim.py index 350a73fbbb..412a7a5089 100644 --- a/tests/avocado/machine_rx_gdbsim.py +++ b/tests/avocado/machine_rx_gdbsim.py @@ -31,6 +31,7 @@ def test_uboot(self): :avocado: tags=3Darch:rx :avocado: tags=3Dmachine:gdbsim-r5f562n8 :avocado: tags=3Dendian:little + :avocado: tags=3Dflaky """ uboot_url =3D ('https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.g= z') uboot_hash =3D '9b78dbd43b40b2526848c0b1ce9de02c24f4dcdb' @@ -56,6 +57,7 @@ def test_linux_sash(self): :avocado: tags=3Darch:rx :avocado: tags=3Dmachine:gdbsim-r5f562n7 :avocado: tags=3Dendian:little + :avocado: tags=3Dflaky """ dtb_url =3D ('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb') dtb_hash =3D '7b4e4e2c71905da44e86ce47adee2210b026ac18' diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/avocado/machi= ne_s390_ccw_virtio.py index 61e75d8f9e..26e938c9e9 100644 --- a/tests/avocado/machine_s390_ccw_virtio.py +++ b/tests/avocado/machine_s390_ccw_virtio.py @@ -167,6 +167,7 @@ def test_s390x_fedora(self): :avocado: tags=3Ddevice:virtio-gpu :avocado: tags=3Ddevice:virtio-crypto :avocado: tags=3Ddevice:virtio-net + :avocado: tags=3Dflaky """ =20 kernel_url =3D ('https://archives.fedoraproject.org/pub/archive' diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py index 0d32cc280e..af086eab08 100644 --- a/tests/avocado/replay_kernel.py +++ b/tests/avocado/replay_kernel.py @@ -88,6 +88,7 @@ def test_x86_64_pc(self): """ :avocado: tags=3Darch:x86_64 :avocado: tags=3Dmachine:pc + :avocado: tags=3Dflaky """ kernel_url =3D ('https://archives.fedoraproject.org/pub/archive/fe= dora' '/linux/releases/29/Everything/x86_64/os/images/pxeb= oot' @@ -186,6 +187,7 @@ def test_arm_cubieboard_initrd(self): """ :avocado: tags=3Darch:arm :avocado: tags=3Dmachine:cubieboard + :avocado: tags=3Dflaky """ deb_url =3D ('https://apt.armbian.com/pool/main/l/' 'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_= armhf.deb') diff --git a/tests/avocado/reverse_debugging.py b/tests/avocado/reverse_deb= ugging.py index 9a468321e5..4cce5a5598 100644 --- a/tests/avocado/reverse_debugging.py +++ b/tests/avocado/reverse_debugging.py @@ -255,6 +255,7 @@ def test_ppc64_pseries(self): """ :avocado: tags=3Darch:ppc64 :avocado: tags=3Dmachine:pseries + :avocado: tags=3Dflaky """ # SLOF branches back to its entry point, which causes this test # to take the 'hit a breakpoint again' path. That's not a problem, @@ -269,6 +270,7 @@ def test_ppc64_powernv(self): """ :avocado: tags=3Darch:ppc64 :avocado: tags=3Dmachine:powernv + :avocado: tags=3Dflaky """ self.endian_is_le =3D False self.reverse_debugging() diff --git a/tests/avocado/smmu.py b/tests/avocado/smmu.py index 05b34418a5..21ff030ca7 100644 --- a/tests/avocado/smmu.py +++ b/tests/avocado/smmu.py @@ -22,6 +22,7 @@ class SMMU(LinuxTest): :avocado: tags=3Dmachine:virt :avocado: tags=3Ddistro:fedora :avocado: tags=3Dsmmu + :avocado: tags=3Dflaky """ =20 IOMMU_ADDON =3D ',iommu_platform=3Don,disable-modern=3Doff,disable-leg= acy=3Don' diff --git a/tests/avocado/tuxrun_baselines.py b/tests/avocado/tuxrun_basel= ines.py index 5f859f4e6f..a936a3b780 100644 --- a/tests/avocado/tuxrun_baselines.py +++ b/tests/avocado/tuxrun_baselines.py @@ -561,6 +561,7 @@ def test_sh4(self): :avocado: tags=3Dimage:zImage :avocado: tags=3Droot:sda :avocado: tags=3Dconsole:ttySC1 + :avocado: tags=3Dflaky """ sums =3D { "rootfs.ext4.zst" : "3592a7a3d5a641e8b9821449e77bc43c9904a56c30d45da0694349cf= d86743fd", --=20 2.39.2