From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007566; cv=none; d=zohomail.com; s=zohoarc; b=VqtQLbOodnByeIlUyfSb0QJqAZzzbEuPNmbGtITSQSN3QMi3EibTtBz7nkTMhKJX4HwuN6uGgow2pE8VllLjAgE11fcVynnRKpKz/yLy6ZHFpNYnd+X5sbBLwJXYK2ExZnIjMn37/clLmZiN9gXk1H583krJ2dT6pg/R+eTfOf4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007566; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KLxxBVgESg8+q++r/+fvnKLfCmcpFTynVpvVjZzQTYc=; b=Il1F+pkwHDjzsuBFFrWkP7N12ciBEgi3tSGDBX4O2GkZGmBfLUb4GLoexIDBLQl6cCdoJ1Y7d0tbhQEAS60tCIcfCKrn3OMb9X0EE8sa+cMO/QWrt5e8EvPnuvwtFGswYyTW6U3Zh9l6U7qeOBoBCZGfRlEvlGmjXsihhknrMRs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159300756613084.51427084047646; Wed, 24 Jun 2020 07:06:06 -0700 (PDT) Received: from localhost ([::1]:33796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo62P-0006V2-6j for importer@patchew.org; Wed, 24 Jun 2020 10:06:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61F-0004m3-OP for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:53 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:37772) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61C-0003Tx-Hc for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:53 -0400 Received: by mail-wr1-x434.google.com with SMTP id a6so2429179wrm.4 for ; Wed, 24 Jun 2020 07:04:50 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 185sm8237788wmz.22.2020.06.24.07.04.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 4C6F91FF87; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KLxxBVgESg8+q++r/+fvnKLfCmcpFTynVpvVjZzQTYc=; b=VwcLfqPbp+T4CAYLNWhwImrkNQUg/MCxrqpgGmWz5g/2C21hymKonLfsM6GwlacWol Q6vbsoWH5gYC+W1TWoRXxM3uM5kJjYeNn79or5nGcho35BNWgsTfmeoE2X7P8F6nR8Ty ts6EncUOKpud+pHpKq7ukXC+zeNjk5fZ2EL/9THYsilF+51TF6krgr7lS7pDTn7HE/mX aiAbFnWBcofwmlVz2vLYZ/+LCpGqy53SQkADa7q07rTQN4j+jn/qE2zIQ/e7kW2kyaXv wfbgf2HqRITx9sYY+aXART2mToePlOOx3Een48LcQh0lS2H9RxTA9VNyzER3pqCMCI62 Egng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KLxxBVgESg8+q++r/+fvnKLfCmcpFTynVpvVjZzQTYc=; b=R0Q3Z76fzD+xW2/j9IwReh7NgDokcF3HjiNHCuA3KjtOCDKy1oKc7NkchPqj/MfU7P m9hxc81tvggHyDwME4XL+a00irVP+NrINjBPTeqQ+py6IwWTt8FNomCyB92gJS2mQLKj Ukw1oupafCLylz63XC6H4PsxJHAlcC0jApDASgen7YFRzXhABpVMTR4g2jFJAFfd/GNj gn+ETC+qe3WqLKYH9v2SlrLywgzI6dX2zLTK1/e+d69aN3sq3jXRzJCV2ihwIB35/By5 V8ImTIlCV/NgY7EEahN3IER0WrWKhtzfE9f6neBEBn/ZT612j43dunZJjzairAq0vj3r hzeA== X-Gm-Message-State: AOAM530GvUn0jhXs3gva5ZrQJ6uf+xWDgLUb46fIFBGCH+ycmBj8774Z GklXK7mZVbjsk5ZQVyqzSuIP/Q== X-Google-Smtp-Source: ABdhPJx6Nn/euRo80F1/ttjqnACDmvz5KtiuijW7KTI5LR2E9CROt4Xt1mpEHUCwC5Rsee7FlmPv6w== X-Received: by 2002:adf:f3cd:: with SMTP id g13mr16503875wrp.45.1593007489056; Wed, 24 Jun 2020 07:04:49 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 01/25] iotests: Fix 051 output after qdev_init_nofail() removal Date: Wed, 24 Jun 2020 15:04:22 +0100 Message-Id: <20200624140446.15380-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::434; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x434.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Kevin Wolf , Thomas Huth , berrange@redhat.com, "open list:Block layer core" , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , richard.henderson@linaro.org, f4bug@amsat.org, Max Reitz , =?UTF-8?q?Alex=20Benn=C3=A9e?= , cota@braap.org, John Snow , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Philippe Mathieu-Daud=C3=A9 Commit 96927c744 replaced qdev_init_nofail() call by isa_realize_and_unref() which has a different error message. Update the test output accordingly. Gitlab CI error after merging b77b5b3dc7: https://gitlab.com/qemu-project/qemu/-/jobs/597414772#L4375 Reported-by: Thomas Huth Signed-off-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Alex Benn=C3=A9e Reviewed-by: John Snow Reviewed-by: Thomas Huth Message-Id: <20200616154949.6586-1-philmd@redhat.com> --- tests/qemu-iotests/051.pc.out | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out index 0ea80d35f0e..da8ad871876 100644 --- a/tests/qemu-iotests/051.pc.out +++ b/tests/qemu-iotests/051.pc.out @@ -142,7 +142,7 @@ QEMU X.Y.Z monitor - type 'help' for more information =20 Testing: -drive if=3Dide QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Device needs med= ia, but drive is empty +(qemu) QEMU_PROG: Device needs media, but drive is empty =20 Testing: -drive if=3Dvirtio QEMU X.Y.Z monitor - type 'help' for more information @@ -214,7 +214,7 @@ QEMU X.Y.Z monitor - type 'help' for more information =20 Testing: -drive file=3DTEST_DIR/t.qcow2,if=3Dide,readonly=3Don QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Block node is re= ad-only +(qemu) QEMU_PROG: Block node is read-only =20 Testing: -drive file=3DTEST_DIR/t.qcow2,if=3Dvirtio,readonly=3Don QEMU X.Y.Z monitor - type 'help' for more information --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007566; cv=none; d=zohomail.com; s=zohoarc; b=GfNA2OU4+0J76GdtmMKHKlPYfYI1otRoxlI4F4MCJW3g0hImiYo23r9QXJz+5jvmc7xRozusDHH+EQ+gnq/ux9rVBi2vAYc/NZryr0g9lzx1KOkaOiNpe7amOBxB7ZrnAVkrCm/T8E4aNkI/gd/9iKXy9YGDY2voGMlkOkAEvaw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007566; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Pp31LDbBMynakzVJX+RF0ypg4zh/bBa29zUb++Htq94=; b=BHNW291+UaL232jds8nnS2GuKSeZ08BjfvZxQzb7UQ8k0IE+D25AM2Q6jTM3KuiXTeN9MGOpRkBMfoXSqEkkjhfX1JIzptXBXpbt+qYpXee3m3fEyayNNRKFb7V+EqIm8tlduvQ8Vcy6Oe4F2rA9m+kfbWzPeYfENVsbZbV1Ixo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007566683613.7885752635883; Wed, 24 Jun 2020 07:06:06 -0700 (PDT) Received: from localhost ([::1]:33864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo62P-0006Wv-P1 for importer@patchew.org; Wed, 24 Jun 2020 10:06:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59056) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61H-0004mk-SF for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:55 -0400 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:52330) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61D-0003U8-JL for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:55 -0400 Received: by mail-wm1-x32a.google.com with SMTP id q15so2441994wmj.2 for ; Wed, 24 Jun 2020 07:04:51 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id v5sm7368959wre.87.2020.06.24.07.04.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:47 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 610051FF8C; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pp31LDbBMynakzVJX+RF0ypg4zh/bBa29zUb++Htq94=; b=D9gmfRWUKxwHohfHoYJsgG+1X0WI91GTOdJ1S8ZlUNDyhayRgm0+kCdETfntKMjro3 4GrakiJ9zoOwBVudoThcL3hGhWWdKP4yS/PpaiIc2fWn9+HWTRD4I5Z01dsMKeUW/knq dyHf1OODjwyNpoCYM6cYobUdIpKR5ARMLo6U6000U+wfbsUqVHaVNvyhSmdbBQUOnnqa caitrgDsHV0+jTl0C4N1mWPggRsKZRfNxg5/NJ5d/Rjf0LFPI1QjsgZcPFAgdlso9tQd RxAihyAghIPfq+72oygX1VmhbRx1nr1Wmz261lzZYs1QFMRkNVVQhwfes5bLyQNqY8zO L28g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pp31LDbBMynakzVJX+RF0ypg4zh/bBa29zUb++Htq94=; b=R5Xjp2HY2VX10wkEnYW0t9tThwhWtIvcBbzigmwUzvT8Mavp+zPMzjytyt5GzQuPO9 lTWJVAdF83jwj8gjBPbP4uorMMVjFEH54K0iqShJzDTTZvlancWJ2WpH484kdYlhhRWi WtCbfPwJ23Iret4jdfUVbFcwcsCLz7fXW3W1lx6BNawImUsqIWi49e9LAzcbd2tewrnX Qp+fiMfHQ0TI2+/1OsZOrzMleblmKXW+eBdLZ7bYyGqAsTXdTPSsuSRz1eHIyzJUBMnD ODAaVnwUp2vC3Sva64xCCGI97xfDAwx+EjpfnnRo4jkCeY6sgXoXOWImlmwkutnokHCz bN5w== X-Gm-Message-State: AOAM531/W9gINAOyevjwV+URmQRZCIY+PXnpWhefzqHUWmirEUGl/pbI OaQTMJmqXVq6j49S6mhMXMgomvU/bmk= X-Google-Smtp-Source: ABdhPJxst9JPfoAEuEt+13DSWD7rj+UJY/SRqwv0cNqBJh8a+lH+KlOUHyzcut07JVTh4VrPz5OTRw== X-Received: by 2002:a1c:de07:: with SMTP id v7mr6471931wmg.56.1593007490169; Wed, 24 Jun 2020 07:04:50 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 02/25] crypto/linux_keyring: fix 'secret_keyring' configure test Date: Wed, 24 Jun 2020 15:04:23 +0100 Message-Id: <20200624140446.15380-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, David Edmondson , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: David Edmondson The configure test for 'secret_keyring' incorrectly checked the 'have_keyring' variable. Fixes: 54e7aac0562452e4fcab65ca5001d030eef2de15 Signed-off-by: David Edmondson Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200618092636.71832-1-david.edmondson@oracle.com> Reviewed-by: Thomas Huth --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure b/configure index ba88fd18244..69ffd750dbc 100755 --- a/configure +++ b/configure @@ -6369,7 +6369,7 @@ EOF fi if test "$secret_keyring" !=3D "no" then - if test "$have_keyring" =3D=3D "yes" + if test "$have_keyring" =3D "yes" then secret_keyring=3Dyes else --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007664; cv=none; d=zohomail.com; s=zohoarc; b=T+NmG37V4gG26eVVXzK6lTPg8qa9XOwDlw55vvc4HkTTrZj2MR/aC/kl3Rn/cYvX0LGEf9OOc7GHNP5cIWhA6nAH+tdoI7ZzGwEENrKsipJcxekPih03bdt2uXLk0EapsukLkWsGct2SUVlG221HIbABUWk8VgFAGnn8R6DuUHg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007664; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=m1Iumf3YfxuO4NhxteboEtLnibs/CAVIrPHuCmkmoi4=; b=aYe/tABUMnhUEd3jV2+gV2QW7m15fcS32SbbUIrSE2LhJ/TIPTbMQaE5u7iO3xwj2dJkJrPjXwhRl3fhe29JN1B/avawVckvO/qvikXHnMULtqxC+/WmMNKLjeLuOuaahbD/adQdDDXhfIRqwap1UZXlQ8vojqr3KPIHD6A2V8k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007664705157.12626080963275; Wed, 24 Jun 2020 07:07:44 -0700 (PDT) Received: from localhost ([::1]:42918 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo63y-0002Dy-Tc for importer@patchew.org; Wed, 24 Jun 2020 10:07:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59104) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61J-0004oh-Gq for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:57 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:40934) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61H-0003UG-JY for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:57 -0400 Received: by mail-wm1-x333.google.com with SMTP id f139so2643732wmf.5 for ; Wed, 24 Jun 2020 07:04:53 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id q128sm8069049wma.38.2020.06.24.07.04.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:47 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 77F151FF8F; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m1Iumf3YfxuO4NhxteboEtLnibs/CAVIrPHuCmkmoi4=; b=kwaweC2bZkw1W/fNIuATNlTX6TQouli102mvlVEYK7WwCuHm7tMnRNz3dxHew5Fi4y 5/hg4d7ZmkbEljEPaXS18F0/1q4R7VLoQLGdmnvJiQ6sDyjZJFSRXfk8FR2PcPoR37p1 JamsUyt/3W2nUCtfx1Kv4Srs3393dtkEJBRecsq1yzzMWEGepFxiOdUsl/V4whKBKzgV wG8+9BQ19JJtcb6aGSztQwb6mp6IEOMQ4AWuF7Hkdz8rkiJBbEaAgqrF+exxxmFqx9tV b8FTPDIQ1FF0z/hDJGd9vHBUUhtLCwvFO0KeBibyHTMb+BBfR5ouaCGKjC61YDTe/rFR ZSqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m1Iumf3YfxuO4NhxteboEtLnibs/CAVIrPHuCmkmoi4=; b=VHl4ObqiK3gwtLG2rEmwk8FuAwfo4q/7GgrC6M4dNGu+lPXkslvzrHHN1X1oVn3Omc vJkDZW4P2VGeReFvhM6+APpC7aa4kpy2NwRGkESYmCNhDPvhYHa+pl2U1pQhaTh6uXRR CfLWDnm4vpFlgpm3W18e+ba2gLDJGb5lMilKJ3O07IMJGIsfSBTvwONKdysFhfCwUu4D PXv8Z4VgvFCybq5zVFCXFOvF6VjdDEW0vNJ/4q9UUoqufH2SonDsGZHRNf3zDAZUZ9rg FM4kneWKzvOqbHG+n0O9PZiDkGe3m7O5hQvSHakX+MCWGXTGmR7PxXWNhQruvaNdchVy Uhvg== X-Gm-Message-State: AOAM530EMRZb3/PTykxN0golaCAiUkf2cG1zsZ7tJUP5hypL7IvDLKz1 0g5ZkuP5vAF3Wo0cxcN9DNBZOHEKuyU= X-Google-Smtp-Source: ABdhPJx3jy1MLaow5X/qWZ9TUytxavu7ZHU5CdYDdGVoTeIu/Y6bNJXwUJJDuWzGUOx7/GKNPWhrEw== X-Received: by 2002:a1c:1d46:: with SMTP id d67mr31968222wmd.152.1593007491419; Wed, 24 Jun 2020 07:04:51 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 03/25] tests/vm: pass args through to BaseVM's __init__ Date: Wed, 24 Jun 2020 15:04:24 +0100 Message-Id: <20200624140446.15380-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x333.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley Adding the args parameter to BaseVM's __init__. We will shortly need to pass more parameters to the class so let's just pass args rather than growing the parameter list. Signed-off-by: Robert Foley Reviewed-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-2-robert.foley@linaro.org> --- tests/vm/basevm.py | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index a80b616a08d..5a58e6c3930 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -61,11 +61,10 @@ class BaseVM(object): # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier =3D 4 - def __init__(self, debug=3DFalse, vcpus=3DNone, genisoimage=3DNone, - build_path=3DNone): + def __init__(self, args): self._guest =3D None - self._genisoimage =3D genisoimage - self._build_path =3D build_path + self._genisoimage =3D args.genisoimage + self._build_path =3D args.build_path self._tmpdir =3D os.path.realpath(tempfile.mkdtemp(prefix=3D"vm-te= st-", suffix=3D".tmp", dir=3D".")) @@ -78,7 +77,7 @@ class BaseVM(object): self._ssh_pub_key_file =3D os.path.join(self._tmpdir, "id_rsa.pub") open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) =20 - self.debug =3D debug + self.debug =3D args.debug self._stderr =3D sys.stderr self._devnull =3D open(os.devnull, "w") if self.debug: @@ -92,8 +91,8 @@ class BaseVM(object): (",ipv6=3Dno" if not self.ipv6 else ""), "-device", "virtio-net-pci,netdev=3Dvnet", "-vnc", "127.0.0.1:0,to=3D20"] - if vcpus and vcpus > 1: - self._args +=3D ["-smp", "%d" % vcpus] + if args.jobs and args.jobs > 1: + self._args +=3D ["-smp", "%d" % args.jobs] if kvm_available(self.arch): self._args +=3D ["-enable-kvm"] else: @@ -456,8 +455,7 @@ def main(vmcls): return 1 logging.basicConfig(level=3D(logging.DEBUG if args.debug else logging.WARN)) - vm =3D vmcls(debug=3Dargs.debug, vcpus=3Dargs.jobs, - genisoimage=3Dargs.genisoimage, build_path=3Dargs.build= _path) + vm =3D vmcls(args) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.im= age, --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007572; cv=none; d=zohomail.com; s=zohoarc; b=nVLXO9xRK2oU+K7FntzoqPTqIGg8zzYXidG3QeWyNVJ28dTMBFce4LRiC8Zg88Z81o2qmHQYPxj9pD0rOj+orxeAoyE6VeGfFL4wlh1D5hOztGb3pWRVk+m8xrcP6IpmuKXtMEeF20I8vtGUslY+UHgQWLjmT6fw5i/YhoPHANQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007572; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Uqu+G7KWNLFFsBFSWuF2fzigErk1o01AVduM2CJF95Y=; b=Wpw6XPET9WdzdancgpT1rnX++L+eaeGE7VbKD0pir2IoTEqnnktOUM6OjCKZpY3FkkEzXpdlASdHU1Pb2N6GZzwem7VSismAjLlm7QuvsOuf2TnCTUgi1+aKVkFIt/byQayMLJsk4s6SD+fzF7CSU6Qh1QlCU5S5GFPCOzy34mo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007572061514.8171107425192; Wed, 24 Jun 2020 07:06:12 -0700 (PDT) Received: from localhost ([::1]:34332 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo62U-0006oO-W6 for importer@patchew.org; Wed, 24 Jun 2020 10:06:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61L-0004t0-Gi for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:59 -0400 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:32850) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61H-0003Ut-LY for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:59 -0400 Received: by mail-wr1-x42b.google.com with SMTP id j94so2448518wrj.0 for ; Wed, 24 Jun 2020 07:04:55 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 65sm23055973wre.6.2020.06.24.07.04.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:50 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 8F78C1FF90; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Uqu+G7KWNLFFsBFSWuF2fzigErk1o01AVduM2CJF95Y=; b=kRb9x49Uqtz+fvIwF5MB3aImZKjEBV3gRH6my7iMox9DI74HOCeYeV4jsF3/aLCiPx RARUxGWqqAj0LVZ+JGQfBdagYnec1H7Is34OkSx1s7rYHESQXst6pVVb45rZ0ZwAe3+i fXYlmFTwzeaewwiNlhaW01IeJxZe1nkpb9nOOkqnLEXqMBkk0IfGBYdFrBmac+8hwQmm DykpeU2TqUzujJ0maiJ2tLjprrYVvp4efKqAX9LWx+TRDzh7DjDr2pgN3WIVdbSKRAOy iO478efuPDto127kFOwMt/hJdkQ/1ZJfy8OAAdp1ZNhOhElefeQz6nN3y320xhA3zb/8 l7Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Uqu+G7KWNLFFsBFSWuF2fzigErk1o01AVduM2CJF95Y=; b=PKlMyJmZRHpW1dmqzYMvM6JM7CoC1gUt0QuJG2DbybWoyfy/xWLDBeKGtHEsJvuVOI Lud40ruSdsI+eXbTUwfid6CCzklbDFeHrrWcXWRDJX1XlYa/4eMrxANGHkGjkxz3GsCr lVQ4lZJFFDldWZqxPmZcHoEO/YcBeqWHagaIm9RPn+e/wrJhRmLfAMwaImP7yJDwGVEv g6K6jV7/nzEAIrz5JHqtVOC6O/diFHoHZdNIaclx8CrhNTya7IJUaxXWNHs2RO1SfLYk uCJPhBAOo0EAWJhudjO+lI6//BUL5BLqgWNqxl2yERFjnp6u6o1ZH0xkG03Exh2Djav2 e8Xg== X-Gm-Message-State: AOAM531t3/9sQf0XYtctzNbfb4UQgk0BhN83v81UPHxW8TDgw9eIm1ul CDaSleHZ1+G8bVg+SyBVcHR1UQ== X-Google-Smtp-Source: ABdhPJyiiOnMPz6gZRNBR+laQtOu2k8wgS/VEMducPjCwPcDKI7ChEpBQy9Tr8h56TeuoEZ0JYauew== X-Received: by 2002:a5d:6a06:: with SMTP id m6mr15411111wru.321.1593007494037; Wed, 24 Jun 2020 07:04:54 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 04/25] tests/vm: Add configuration to basevm.py Date: Wed, 24 Jun 2020 15:04:25 +0100 Message-Id: <20200624140446.15380-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley Added use of a configuration to tests/vm/basevm.py. The configuration provides parameters used to configure a VM. This allows for providing alternate configurations to the VM being created/launched. cpu, machine, memory, and NUMA configuration are all examples of configuration which we might want to vary on the VM being creat= ed or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Benn=C3=A9e Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-3-robert.foley@linaro.org> --- tests/vm/basevm.py | 172 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 133 insertions(+), 39 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 5a58e6c3930..cfe20c58f7e 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -29,16 +29,41 @@ import tempfile import shutil import multiprocessing import traceback - -SSH_KEY =3D open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa")).read() -SSH_PUB_KEY =3D open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa.pub")).read() - +import shlex + +SSH_KEY_FILE =3D os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa") +SSH_PUB_KEY_FILE =3D os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa.pub") + +# This is the standard configuration. +# Any or all of these can be overridden by +# passing in a config argument to the VM constructor. +DEFAULT_CONFIG =3D { + 'cpu' : "max", + 'machine' : 'pc', + 'guest_user' : "qemu", + 'guest_pass' : "qemupass", + 'root_pass' : "qemupass", + 'ssh_key_file' : SSH_KEY_FILE, + 'ssh_pub_key_file': SSH_PUB_KEY_FILE, + 'memory' : "4G", + 'extra_args' : [], + 'qemu_args' : "", + 'dns' : "", + 'ssh_port' : 0, + 'install_cmds' : "", + 'boot_dev_type' : "block", + 'ssh_timeout' : 1, +} +BOOT_DEVICE =3D { + 'block' : "-drive file=3D{},if=3Dnone,id=3Ddrive0,cache=3Dwriteback "\ + "-device virtio-blk,drive=3Ddrive0,bootindex=3D0", + 'scsi' : "-device virtio-scsi-device,id=3Dscsi "\ + "-drive file=3D{},format=3Draw,if=3Dnone,id=3Dhd0 "\ + "-device scsi-hd,drive=3Dhd0,bootindex=3D0", +} class BaseVM(object): - GUEST_USER =3D "qemu" - GUEST_PASS =3D "qemupass" - ROOT_PASS =3D "qemupass" =20 envvars =3D [ "https_proxy", @@ -57,25 +82,38 @@ class BaseVM(object): poweroff =3D "poweroff" # enable IPv6 networking ipv6 =3D True + # This is the timeout on the wait for console bytes. + socket_timeout =3D 120 # Scale up some timeouts under TCG. # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier =3D 4 - def __init__(self, args): + def __init__(self, args, config=3DNone): self._guest =3D None self._genisoimage =3D args.genisoimage self._build_path =3D args.build_path + # Allow input config to override defaults. + self._config =3D DEFAULT_CONFIG.copy() + if config !=3D None: + self._config.update(config) + self.validate_ssh_keys() self._tmpdir =3D os.path.realpath(tempfile.mkdtemp(prefix=3D"vm-te= st-", suffix=3D".tmp", dir=3D".")) atexit.register(shutil.rmtree, self._tmpdir) - - self._ssh_key_file =3D os.path.join(self._tmpdir, "id_rsa") - open(self._ssh_key_file, "w").write(SSH_KEY) - subprocess.check_call(["chmod", "600", self._ssh_key_file]) - - self._ssh_pub_key_file =3D os.path.join(self._tmpdir, "id_rsa.pub") - open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) + # Copy the key files to a temporary directory. + # Also chmod the key file to agree with ssh requirements. + self._config['ssh_key'] =3D \ + open(self._config['ssh_key_file']).read().rstrip() + self._config['ssh_pub_key'] =3D \ + open(self._config['ssh_pub_key_file']).read().rstrip() + self._ssh_tmp_key_file =3D os.path.join(self._tmpdir, "id_rsa") + open(self._ssh_tmp_key_file, "w").write(self._config['ssh_key']) + subprocess.check_call(["chmod", "600", self._ssh_tmp_key_file]) + + self._ssh_tmp_pub_key_file =3D os.path.join(self._tmpdir, "id_rsa.= pub") + open(self._ssh_tmp_pub_key_file, + "w").write(self._config['ssh_pub_key']) =20 self.debug =3D args.debug self._stderr =3D sys.stderr @@ -84,11 +122,14 @@ class BaseVM(object): self._stdout =3D sys.stdout else: self._stdout =3D self._devnull + netdev =3D "user,id=3Dvnet,hostfwd=3D:127.0.0.1:{}-:22" self._args =3D [ \ - "-nodefaults", "-m", "4G", - "-cpu", "max", - "-netdev", "user,id=3Dvnet,hostfwd=3D:127.0.0.1:0-:22" + - (",ipv6=3Dno" if not self.ipv6 else ""), + "-nodefaults", "-m", self._config['memory'], + "-cpu", self._config['cpu'], + "-netdev", + netdev.format(self._config['ssh_port']) + + (",ipv6=3Dno" if not self.ipv6 else "") + + (",dns=3D" + self._config['dns'] if self._config['dns'] else "= "), "-device", "virtio-net-pci,netdev=3Dvnet", "-vnc", "127.0.0.1:0,to=3D20"] if args.jobs and args.jobs > 1: @@ -99,6 +140,55 @@ class BaseVM(object): logging.info("KVM not available, not using -enable-kvm") self._data_args =3D [] =20 + if self._config['qemu_args'] !=3D None: + qemu_args =3D self._config['qemu_args'] + qemu_args =3D qemu_args.replace('\n',' ').replace('\r','') + # shlex groups quoted arguments together + # we need this to keep the quoted args together for when + # the QEMU command is issued later. + args =3D shlex.split(qemu_args) + self._config['extra_args'] =3D [] + for arg in args: + if arg: + # Preserve quotes around arguments. + # shlex above takes them out, so add them in. + if " " in arg: + arg =3D '"{}"'.format(arg) + self._config['extra_args'].append(arg) + + def validate_ssh_keys(self): + """Check to see if the ssh key files exist.""" + if 'ssh_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_key_file']): + raise Exception("ssh key file not found.") + if 'ssh_pub_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_pub_key_file']): + raise Exception("ssh pub key file not found.") + + def wait_boot(self, wait_string=3DNone): + """Wait for the standard string we expect + on completion of a normal boot. + The user can also choose to override with an + alternate string to wait for.""" + if wait_string is None: + if self.login_prompt is None: + raise Exception("self.login_prompt not defined") + wait_string =3D self.login_prompt + # Intentionally bump up the default timeout under TCG, + # since the console wait below takes longer. + timeout =3D self.socket_timeout + if not kvm_available(self.arch): + timeout *=3D 8 + self.console_init(timeout=3Dtimeout) + self.console_wait(wait_string) + + def __getattr__(self, name): + # Support direct access to config by key. + # for example, access self._config['cpu'] by self.cpu + if name.lower() in self._config.keys(): + return self._config[name.lower()] + return object.__getattribute__(self, name) + def _download_with_cache(self, url, sha256sum=3DNone, sha512sum=3DNone= ): def check_sha256sum(fname): if not sha256sum: @@ -130,8 +220,9 @@ class BaseVM(object): "-t", "-o", "StrictHostKeyChecking=3Dno", "-o", "UserKnownHostsFile=3D" + os.devnull, - "-o", "ConnectTimeout=3D1", - "-p", self.ssh_port, "-i", self._ssh_key_file] + "-o", + "ConnectTimeout=3D{}".format(self._config["ssh_timeout"= ]), + "-p", self.ssh_port, "-i", self._ssh_tmp_key_file] # If not in debug mode, set ssh to quiet mode to # avoid printing the results of commands. if not self.debug: @@ -180,14 +271,14 @@ class BaseVM(object): "virtio-blk,drive=3D%s,serial=3D%s,bootindex= =3D1" % (name, name)] =20 def boot(self, img, extra_args=3D[]): - args =3D self._args + [ - "-drive", "file=3D%s,if=3Dnone,id=3Ddrive0,cache=3Dwriteback" = % img, - "-device", "virtio-blk,drive=3Ddrive0,bootindex=3D0"] - args +=3D self._data_args + extra_args + boot_dev =3D BOOT_DEVICE[self._config['boot_dev_type']] + boot_params =3D boot_dev.format(img) + args =3D self._args + boot_params.split(' ') + args +=3D self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_path =3D get_qemu_path(self.arch, self._build_path) guest =3D QEMUMachine(binary=3Dqemu_path, args=3Dargs) - guest.set_machine('pc') + guest.set_machine(self._config['machine']) guest.set_console() try: guest.launch() @@ -301,7 +392,8 @@ class BaseVM(object): self.console_send(command) =20 def console_ssh_init(self, prompt, user, pw): - sshkey_cmd =3D "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.= rstrip() + sshkey_cmd =3D "echo '%s' > .ssh/authorized_keys\n" \ + % self._config['ssh_pub_key'].rstrip() self.console_wait_send("login:", "%s\n" % user) self.console_wait_send("Password:", "%s\n" % pw) self.console_wait_send(prompt, "mkdir .ssh\n") @@ -360,23 +452,23 @@ class BaseVM(object): "local-hostname: {}-guest\n".format(name)]) mdata.close() udata =3D open(os.path.join(cidir, "user-data"), "w") - print("guest user:pw {}:{}".format(self.GUEST_USER, - self.GUEST_PASS)) + print("guest user:pw {}:{}".format(self._config['guest_user'], + self._config['guest_pass'])) udata.writelines(["#cloud-config\n", "chpasswd:\n", " list: |\n", - " root:%s\n" % self.ROOT_PASS, - " %s:%s\n" % (self.GUEST_USER, - self.GUEST_PASS), + " root:%s\n" % self._config['root_pass'], + " %s:%s\n" % (self._config['guest_user'], + self._config['guest_pass']), " expire: False\n", "users:\n", - " - name: %s\n" % self.GUEST_USER, + " - name: %s\n" % self._config['guest_user'], " sudo: ALL=3D(ALL) NOPASSWD:ALL\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], " - name: root\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], "locale: en_US.UTF-8\n"]) proxy =3D os.environ.get("http_proxy") if not proxy is None: @@ -447,15 +539,17 @@ def parse_args(vmcls): parser.disable_interspersed_args() return parser.parse_args() =20 -def main(vmcls): +def main(vmcls, config=3DNone): try: + if config =3D=3D None: + config =3D {} args, argv =3D parse_args(vmcls) if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 logging.basicConfig(level=3D(logging.DEBUG if args.debug else logging.WARN)) - vm =3D vmcls(args) + vm =3D vmcls(args, config=3Dconfig) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.im= age, --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007830; cv=none; d=zohomail.com; s=zohoarc; b=htiMuZR/vV257cfhLhttc+VMAZP0ftxkZuOga/xGrVK4BHGMqhs3BDrk4TrrAUVVpG45Oa79s49FF9xufGBIWprCe9nz9qjsDT0ZyfQ5Wtjf/CuRwIx5d0fo7kWp3UJfm0TaMTcr1OBI/KyXwAH8ipXosFvE4S52dMqv6izKY2Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007830; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KjVUPk/VZOkBn/Ptl1C1G/mY93IJVmlYmN70Nj+SmGE=; b=XbPD+9bLy9IbwIqLkolIbGPmJz24AYsHisxOggQpIL7Ef5u6mzKEUUA74piTe2O/21xckT6mCkudbCmIKS9PoPkZgXoKYKlF8MgVkzvf/PacafQWAzlnklxiYh/0OC0vCb9TzIjKa+K9vI0F9+HKIGjIcM+6zFOYwWfRAFScsQE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007830819949.000784763563; Wed, 24 Jun 2020 07:10:30 -0700 (PDT) Received: from localhost ([::1]:56686 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo66f-0000AB-Ud for importer@patchew.org; Wed, 24 Jun 2020 10:10:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59214) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61O-0004ya-PB for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:02 -0400 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]:37580) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61K-0003VL-84 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:02 -0400 Received: by mail-wm1-x341.google.com with SMTP id o2so2656340wmh.2 for ; Wed, 24 Jun 2020 07:04:57 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id k126sm2719576wme.17.2020.06.24.07.04.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:51 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id ABFD61FF91; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KjVUPk/VZOkBn/Ptl1C1G/mY93IJVmlYmN70Nj+SmGE=; b=FolOEAN8ONAYAXa/5Tn3d0WoK8TDgC3vjSPzSiNColv5bTr9B/4WQzWDfsnPHcENne EiIVxi9eMlwu7v4uh8cRLNMmUCZ4YLzJbAEGRM0srXQwkeCPUZIKkzbOiXabtRg54/H2 baeDbVO4GBEFv7u8Mf+02fVnZIcCC1b7SkluF+aJA3a1ZWSIGnAFNIXiwafeajZlJU2R PVp9jvSRNz0ixgFj6TUK3vJMjIg4iOSwigTPmDQKJlBhpvcGun2fFrRKismQKp4/1U2d pMWiLgIIS1oraDQa5RqxVI9qfrdJTD68uz/WM07XjuUbPqNcvjkaCRmpJc+1t73yzk1A Kgsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KjVUPk/VZOkBn/Ptl1C1G/mY93IJVmlYmN70Nj+SmGE=; b=PCpoOKzGD9i+E9yAZSyjFvlP7qZw3XmTt4wb38Y4/5kfMmDWy6vuZmk3ls2Z5e/i+S Hb6f8GtKMfl+whapXe5uhUW7Y+4L6+rz/XmKdVEMwph5tTMUXvuEIh2Iwznqgec5ev/5 hfiMc+Zv4fRP36uLFDhkGze4am0Ts5kuiZ84Y5R9BPu0IDYJNUvWty2QPyTkhTb/Xgye fKgzVDSX6ohpqZTCmw0y1HqMDIfC8lMyE7xPjbv3Dx8ujMphqD8Z+Q02v8FHWuqSRHOm aUoQPqq++hLiQKfAv/NIr/4McyP2sn7xTGBHFQPpfGHdxUH9FrhfGlxzFHmtl5qhvdBt hmXw== X-Gm-Message-State: AOAM530G2qUmsvp6jH0Q3qPjhCurJCta0jQ+OW3R7HONKgL5WFPHGEnD zpkLV2u9+X8yW5XNEpajKK0QdA== X-Google-Smtp-Source: ABdhPJxEBmFXzgOKJxrREOmI0/cPbxs1hi2Ac1y4J8uMzoNShCxp/VViA/qQViUM2NUITLuRfilS+w== X-Received: by 2002:a1c:6788:: with SMTP id b130mr30108151wmc.100.1593007496604; Wed, 24 Jun 2020 07:04:56 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 05/25] tests/vm: Added configuration file support Date: Wed, 24 Jun 2020 15:04:26 +0100 Message-Id: <20200624140446.15380-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::341; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x341.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley Changes to tests/vm/basevm.py to allow accepting a configuration file as a parameter. Allows for specifying VM options such as cpu, machine, memory, and arbitrary qemu arguments for specifying options such as NUMA configuration. Also added an example conf_example_aarch64.yml and conf_example_x86.yml. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Benn=C3=A9e Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-4-robert.foley@linaro.org> --- configure | 9 ++++++ tests/vm/Makefile.include | 6 ++++ tests/vm/basevm.py | 40 +++++++++++++++++++++++- tests/vm/conf_example_aarch64.yml | 51 +++++++++++++++++++++++++++++++ tests/vm/conf_example_x86.yml | 50 ++++++++++++++++++++++++++++++ 5 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 tests/vm/conf_example_aarch64.yml create mode 100644 tests/vm/conf_example_x86.yml diff --git a/configure b/configure index 69ffd750dbc..c1ec271931f 100755 --- a/configure +++ b/configure @@ -958,6 +958,13 @@ do fi done =20 +# Check for existence of python3 yaml, needed to +# import yaml config files into vm-build. +python_yaml=3D"no" +if $(python3 -c "import yaml" 2> /dev/null); then + python_yaml=3D"yes" +fi + : ${smbd=3D${SMBD-/usr/sbin/smbd}} =20 # Default objcc to clang if available, otherwise use CC @@ -6751,6 +6758,7 @@ if test "$docs" !=3D "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" !=3D "no" ; then echo "smbd $smbd" @@ -7830,6 +7838,7 @@ echo "PYTHON=3D$python" >> $config_host_mak echo "SPHINX_BUILD=3D$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=3D$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=3D$genisoimage" >> $config_host_mak +echo "PYTHON_YAML=3D$python_yaml" >> $config_host_mak echo "CC=3D$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then echo "IASL=3D$iasl" >> $config_host_mak diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index a253aba4579..f6c3892bb28 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -44,6 +44,12 @@ endif @echo " QEMU_LOCAL=3D1 - Use QEMU binary local to this= build." @echo " QEMU=3D/path/to/qemu - Change path to QEMU binary" @echo " QEMU_IMG=3D/path/to/qemu-img - Change path to qemu-img tool" +ifeq ($(PYTHON_YAML),yes) + @echo " QEMU_CONFIG=3D/path/conf.yml - Change path to VM configurati= on .yml file." +else + @echo " (install python3-yaml to enable support for yaml file to confi= gure a VM.)" +endif + @echo " See conf_example_*.yml for file= format details." =20 vm-build-all: $(addprefix vm-build-, $(IMAGES)) =20 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index cfe20c58f7e..fa56fbbb4b6 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -481,7 +481,6 @@ class BaseVM(object): cwd=3Dcidir, stdin=3Dself._devnull, stdout=3Dself._stdout, stderr=3Dself._stdout) - return os.path.join(cidir, "cloud-init.iso") =20 def get_qemu_path(arch, build_path=3DNone): @@ -497,6 +496,41 @@ def get_qemu_path(arch, build_path=3DNone): qemu_path =3D "qemu-system-" + arch return qemu_path =20 +def parse_config(config, args): + """ Parse yaml config and populate our config structure. + The yaml config allows the user to override the + defaults for VM parameters. In many cases these + defaults can be overridden without rebuilding the VM.""" + if args.config: + config_file =3D args.config + elif 'QEMU_CONFIG' in os.environ: + config_file =3D os.environ['QEMU_CONFIG'] + else: + return config + if not os.path.exists(config_file): + raise Exception("config file {} does not exist".format(config_file= )) + # We gracefully handle importing the yaml module + # since it might not be installed. + # If we are here it means the user supplied a .yml file, + # so if the yaml module is not installed we will exit with error. + try: + import yaml + except ImportError: + print("The python3-yaml package is needed "\ + "to support config.yaml files") + # Instead of raising an exception we exit to avoid + # a raft of messy (expected) errors to stdout. + exit(1) + with open(config_file) as f: + yaml_dict =3D yaml.safe_load(f) + + if 'qemu-conf' in yaml_dict: + config.update(yaml_dict['qemu-conf']) + else: + raise Exception("config file {} is not valid"\ + " missing qemu-conf".format(config_file)) + return config + def parse_args(vmcls): =20 def get_default_jobs(): @@ -536,6 +570,9 @@ def parse_args(vmcls): help=3D"run tests with a snapshot") parser.add_option("--genisoimage", default=3D"genisoimage", help=3D"iso imaging tool") + parser.add_option("--config", "-c", default=3DNone, + help=3D"Provide config yaml for configuration. "\ + "See config_example.yaml for example.") parser.disable_interspersed_args() return parser.parse_args() =20 @@ -547,6 +584,7 @@ def main(vmcls, config=3DNone): if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 + config =3D parse_config(config, args) logging.basicConfig(level=3D(logging.DEBUG if args.debug else logging.WARN)) vm =3D vmcls(args, config=3Dconfig) diff --git a/tests/vm/conf_example_aarch64.yml b/tests/vm/conf_example_aarc= h64.yml new file mode 100644 index 00000000000..9d44ae356f7 --- /dev/null +++ b/tests/vm/conf_example_aarch64.yml @@ -0,0 +1,51 @@ +# +# Example yaml for use by any of the scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defa= ults. + + # Login username and password(has to be sudo enabled) + guest_user: qemu + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # If one key is provided, both must be provided. + #ssh_key: /complete/path/of/your/keyfile/id_rsa + #ssh_pub_key: /complete/path/of/your/keyfile/id_rsa.pub + + cpu: max + machine: virt,gic-version=3Dmax + memory: 16G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=3D16,sockets=3D2,cores=3D8 + -numa node,cpus=3D0-3,nodeid=3D0 -numa node,cpus=3D4-7,nod= eid=3D1 + -numa node,cpus=3D8-11,nodeid=3D2 -numa node,cpus=3D12-15,= nodeid=3D3 + -numa dist,src=3D0,dst=3D1,val=3D15 -numa dist,src=3D2,dst= =3D3,val=3D15 + -numa dist,src=3D0,dst=3D2,val=3D20 -numa dist,src=3D0,dst= =3D3,val=3D20 + -numa dist,src=3D1,dst=3D2,val=3D20 -numa dist,src=3D1,dst= =3D3,val=3D20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: 1.234.567.89 + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + #ssh_port: 5555 + + # To install a different set of packages, provide a command to issue + #install_cmds: "apt-get update ; apt-get build-dep -y qemu" + + # Or to skip the install entirely, just provide "" + #install_cmds: "" diff --git a/tests/vm/conf_example_x86.yml b/tests/vm/conf_example_x86.yml new file mode 100644 index 00000000000..78d3f5830fa --- /dev/null +++ b/tests/vm/conf_example_x86.yml @@ -0,0 +1,50 @@ +# +# Example yaml for use by any of the x86 based scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defa= ults. + + # Login username and password(has to be sudo enabled) + guest_user: "qemu" + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # Provide default ssh keys of current user. + # You need to edit the below for your user. + #ssh_key_file: /home//.ssh/id_rsa + #ssh_pub_key_file: /home//.ssh/id_rsa.pub + + cpu: max + machine: pc + memory: 8G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=3D8,sockets=3D2,cores=3D4 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D0,id=3Dram-node0 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D0,id=3Dram-node1 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D1,id=3Dram-node2 + -object memory-backend-ram,size=3D4G,policy=3Dbind,host-no= des=3D1,id=3Dram-node3 + -numa node,cpus=3D0-1,nodeid=3D0 -numa node,cpus=3D2-3,nod= eid=3D1 + -numa node,cpus=3D4-5,nodeid=3D2 -numa node,cpus=3D6-7,nod= eid=3D3 + -numa dist,src=3D0,dst=3D1,val=3D15 -numa dist,src=3D2,dst= =3D3,val=3D15 + -numa dist,src=3D0,dst=3D2,val=3D20 -numa dist,src=3D0,dst= =3D3,val=3D20 + -numa dist,src=3D1,dst=3D2,val=3D20 -numa dist,src=3D1,dst= =3D3,val=3D20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: "1.234.567.89" + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + ssh_port: 5555 --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007658; cv=none; d=zohomail.com; s=zohoarc; b=KNGsNIvVOdTpmF+/9XBnm9a3o0VXhev0faeqo0BEpxrhEToc0JIlm9yL0JkmtYe4e7w3pdI+y+97pC94EFgJcqFljgF4Z5Tr259uhx7deO692SRvM34vRUBGxQ1YHMBU6mXaYs1noUkwits5URR6CC9OoVuJtoOwdJO9FuNoibE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007658; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yfaWvkBHiSQ15nCnLOaiCfwwNxNyBgCuiCJ/EYYdU+8=; b=Y26vuzytLkPFm1ce7aeZsh5dx9py1vKSxCthtLLAkEmxgPej6HJmMf6f+ob+SB3hiNVYeg2qULS9ozaG9F3xG+ujgtubqwGTQbasTI5cSJspIt1Zn4zpNUN1S14WyCke0j9pMI1YcJ0U/QkbfPdB4tafp5R4Mo5s3wpPjFz/2XM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15930076582381022.0313172087663; Wed, 24 Jun 2020 07:07:38 -0700 (PDT) Received: from localhost ([::1]:42292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo63t-0001yb-AN for importer@patchew.org; Wed, 24 Jun 2020 10:07:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61M-0004uO-6D for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:00 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:44986) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61I-0003V5-QY for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:59 -0400 Received: by mail-wr1-x434.google.com with SMTP id b6so2392890wrs.11 for ; Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id y25sm8232106wma.19.2020.06.24.07.04.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:51 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id C30CE1FF92; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yfaWvkBHiSQ15nCnLOaiCfwwNxNyBgCuiCJ/EYYdU+8=; b=hydnMo+0y0ZC++K80GMn9ZR6OJ1W8xU5OuMxzDiD6fkNE4XFTiP+DhjJhjPi/L3vzn tr0Da1AbSDCTb2QGgrhxRtrZEgyn2GLBPf9oBommnvINOYKrZ9czErJhLXCqHxI/KT+L 6rIKpXsLC1u5kb7p+LmojWs01r2ACvKYVY03CgFE+kqJ+b1eoVatEumrjVrrGMNxSCn0 8ECEllraXFAatAy7UlcOyQCJ42l4jdC8/r9SfGlWhQNq775m3HIAB97DGLTRqfAObTE5 gdCST3z1AihIUzxucqDZW8IGYu3cLawcNpz0EGxda3CStDjJjtfahlOsE40b1rtsVQcx nrqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yfaWvkBHiSQ15nCnLOaiCfwwNxNyBgCuiCJ/EYYdU+8=; b=jBIuDX2TJEBkOrNxqi8vYr+VuVrwVioNkjfhvO3zsN0w9s7BwPZNZ1APRIrYj8Ubhz Nt+SsHMDivs3CEwzMwEMXITXgZdW+2ZoOXTQ25EynINbN+1f+0McQZtv0sZRsaRLUybU GQ6OoDOL+sI0W9LXHXetOV5tk5I/Z12llt5ryMWIRtU2vt3x3/JO92qa2bYAOOMkhjZR Novaa3PWRENwAgif/rn496wp3arFBaoPn972KFzrrQTAb2SN5F0bhu/Y2nS9Q56zY2C9 6LsqDXiSJ1baYJIAVFuP109syzaBNIGdhHEoehSXObSaIvwhCHsnRKsPCzZuICDGky0j s66A== X-Gm-Message-State: AOAM531XZWIPmzpjZ5ZTREINPgXzZPaRLNeM+Nvlh1kawXnp91g9ihhP W77xtJuyC3kKrWKAPAa5COJF5g== X-Google-Smtp-Source: ABdhPJwQ1xYXA8S0TyyvNEBygrsxP5ZhTifUHtQtQvq/+fZsM4XtDbvFe1Plp4YZ29A8ilqAfZ9a2A== X-Received: by 2002:adf:8067:: with SMTP id 94mr29864433wrk.427.1593007495439; Wed, 24 Jun 2020 07:04:55 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 06/25] tests/vm: Add common Ubuntu python module Date: Wed, 24 Jun 2020 15:04:27 +0100 Message-Id: <20200624140446.15380-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::434; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x434.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley Add a common Ubuntu python module and make use of it with the ubuntu.i386 script. This is preparation for adding an Ubuntu script ubuntu.aarch64. Splitting out the common logic such as build_image() will reduce duplication. Signed-off-by: Robert Foley Tested-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-5-robert.foley@linaro.org> --- tests/vm/ubuntu.i386 | 46 +++++++++------------------------ tests/vm/ubuntuvm.py | 60 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+), 34 deletions(-) create mode 100644 tests/vm/ubuntuvm.py diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 index 24527cc78c7..5ce72610a6c 100755 --- a/tests/vm/ubuntu.i386 +++ b/tests/vm/ubuntu.i386 @@ -11,15 +11,22 @@ # the COPYING file in the top-level directory. # =20 -import os import sys -import subprocess import basevm -import time +import ubuntuvm =20 -class UbuntuX86VM(basevm.BaseVM): +DEFAULT_CONFIG =3D { + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y qemu,"\ + "apt-get install -y libfdt-dev language-pack-en", +} + +class UbuntuX86VM(ubuntuvm.UbuntuVM): name =3D "ubuntu.i386" arch =3D "i386" + image_link=3D"https://cloud-images.ubuntu.com/releases/bionic/"\ + "release-20191114/ubuntu-18.04-server-cloudimg-i386.img" + image_sha256=3D"28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37a= c4b4ff04ef" BUILD_SCRIPT =3D """ set -e; cd $(mktemp -d); @@ -29,34 +36,5 @@ class UbuntuX86VM(basevm.BaseVM): make --output-sync {target} -j{jobs} {verbose}; """ =20 - def build_image(self, img): - cimg =3D self._download_with_cache( - "https://cloud-images.ubuntu.com/releases/bionic/release-20191= 114/ubuntu-18.04-server-cloudimg-i386.img", - sha256sum=3D"28969840626d1ea80bb249c08eef1a4533e8904aa51a327b4= 0f37ac4b4ff04ef") - img_tmp =3D img + ".tmp" - subprocess.check_call(["cp", "-f", cimg, img_tmp]) - self.exec_qemu_img("resize", img_tmp, "50G") - self.boot(img_tmp, extra_args =3D [ - "-device", "VGA", - "-cdrom", self.gen_cloud_init_iso() - ]) - self.wait_ssh() - self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") - self.ssh_root_check("apt-get update") - self.ssh_root_check("apt-get install -y cloud-initramfs-growroot") - # Don't check the status in case the guest hang up too quickly - self.ssh_root("sync && reboot") - time.sleep(5) - self.wait_ssh() - # The previous update sometimes doesn't survive a reboot, so do it= again - self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sour= ces.list") - self.ssh_root_check("apt-get update") - self.ssh_root_check("apt-get build-dep -y qemu") - self.ssh_root_check("apt-get install -y libfdt-dev language-pack-e= n") - self.ssh_root("poweroff") - self.wait() - os.rename(img_tmp, img) - return 0 - if __name__ =3D=3D "__main__": - sys.exit(basevm.main(UbuntuX86VM)) + sys.exit(basevm.main(UbuntuX86VM, DEFAULT_CONFIG)) diff --git a/tests/vm/ubuntuvm.py b/tests/vm/ubuntuvm.py new file mode 100644 index 00000000000..6689ad87aa8 --- /dev/null +++ b/tests/vm/ubuntuvm.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +# +# Ubuntu VM testing library +# +# Copyright 2017 Red Hat Inc. +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. + +import os +import subprocess +import basevm + +class UbuntuVM(basevm.BaseVM): + + def __init__(self, args, config=3DNone): + self.login_prompt =3D "ubuntu-{}-guest login:".format(self.arch) + basevm.BaseVM.__init__(self, args, config) + + def build_image(self, img): + """Build an Ubuntu VM image. The child class will + define the install_cmds to init the VM.""" + os_img =3D self._download_with_cache(self.image_link, + sha256sum=3Dself.image_sha256) + img_tmp =3D img + ".tmp" + subprocess.check_call(["cp", "-f", os_img, img_tmp]) + self.exec_qemu_img("resize", img_tmp, "+50G") + ci_img =3D self.gen_cloud_init_iso() + + self.boot(img_tmp, extra_args =3D [ "-device", "VGA", "-cdrom", ci= _img, ]) + + # First command we issue is fix for slow ssh login. + self.wait_ssh(wait_root=3DTrue, + cmd=3D"chmod -x /etc/update-motd.d/*") + # Wait for cloud init to finish + self.wait_ssh(wait_root=3DTrue, + cmd=3D"ls /var/lib/cloud/instance/boot-finished") + self.ssh_root("touch /etc/cloud/cloud-init.disabled") + # Disable auto upgrades. + # We want to keep the VM system state stable. + self.ssh_root('sed -ie \'s/"1"/"0"/g\' '\ + '/etc/apt/apt.conf.d/20auto-upgrades') + self.ssh_root("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.li= st") + + # If the user chooses not to do the install phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] !=3D "": + # Issue the install commands. + # This can be overriden by the user in the config .yml. + install_cmds =3D self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.graceful_shutdown() + os.rename(img_tmp, img) + return 0 --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007920; cv=none; d=zohomail.com; s=zohoarc; b=JSsNr70xO8KODh0Y3Uqwwg8Jq6pSLzx2gEQA4PMziwsk6MQcg9KykExQY0uDG0D/LrYxF2tlCNnTUtN+TAznvRCuOEZxRydFyGYBD3XKUCH9kqgnQGYw9BU2k9d05Z6dc71b5xMX/CpxWEZ0TC9Csxo1ZYpToksivhyjBeGHQRU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007920; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eHE1ZGUQed0VLFEhjLciTrVecX1BhNTXwaA8+xC9Diw=; b=m3PxLMnS3AOpugsjRyrK8asPE8zsUBOIH25vRCahhdS3JRTbdqcJIyBKcsdo2U03TWlReOb9CEkhxq4qYR3G1rmekEbcYW6sPYaLJ/q0z+J6BcrZLMGoRLh/+Q5AYw4DRJFy+WwDc2aomSdeHuUyPlObsKowPQk828uDPqqyYHc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007920930261.63779169607824; Wed, 24 Jun 2020 07:12:00 -0700 (PDT) Received: from localhost ([::1]:32798 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo687-00035z-Va for importer@patchew.org; Wed, 24 Jun 2020 10:12:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59234) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61P-00050B-Km for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:03 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:35993) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61L-0003Vd-SQ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:03 -0400 Received: by mail-wm1-x333.google.com with SMTP id 17so2664461wmo.1 for ; Wed, 24 Jun 2020 07:04:59 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n189sm8178049wmb.43.2020.06.24.07.04.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:53 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id E03021FF93; Wed, 24 Jun 2020 15:04:46 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eHE1ZGUQed0VLFEhjLciTrVecX1BhNTXwaA8+xC9Diw=; b=dcgPp4/FWU4YXejrJvdHvSf8UQehwYLLrHLPg19rXnm8UnS0tdGrF3NWhENQiqqI9S 1nuwS5fDUPIMUgsMzjmo67aB3HrQsDiuCYBw4CPY8wyxambMV6H+hP+QsdYQ6/cqD/Qh DpfpMvY1rSRJISRXcZEp1wpkvefqnW71owdtYnXnAmhNJh2tdL4yEiCARp9mIFTeK9t7 XNmZ/Gc0BxK1l5e12xOuHRzKHv56jjk+VsBSlGg7TQmDYXS7kJce0N9Oq/SRdfjjgIto Yr3/hSXbIvtbVmb+BO5REL8gXKkpOCAmgxeg+VVs9QrBiFTiC+Tey+MBSoV7KSJ/KQzF 9QtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eHE1ZGUQed0VLFEhjLciTrVecX1BhNTXwaA8+xC9Diw=; b=VURgE9vc+VWoZb9onRGlJmfbeDgyIFkuWJKfbx12k6GFAeZPTQIF9D6o4HZE5bttTV 4dBp1WdAssF6EyRSKfz/qK9b3aQtH7MfnRJ2kmtTCwXSb1+00f3SqfCPwpfLibrJH4e5 +28e5j9lqIY3T7qPeWkX3jJcMWQPacW7rO/Lb4flqDYduWlOxsr5i8qhE/0wdQbt6BrC t5FIyodwZyiafPwIsM9IW041drSXTiGfPsqQGDQVXeUqhTbQCc1P0Yhm9/rwrei9SieJ JA/nTu964qUPfP4S5P8D19bbFrRQjDcUUYR9dfu28g6aTiCaB+cVJO1H3afMsiB2mdUN 4mMA== X-Gm-Message-State: AOAM532Fd8jNgHSp1AX9Z1g3usD2TWYrHmAetLm/HYbsfIKQ9igbEmv/ xrUqzXvhDqkelrUzu66xiBZB7Q== X-Google-Smtp-Source: ABdhPJxsdhi1XPFTsVTe9HVk96yYmpRiF+L14qT3EU9u3Dd+zb/QbMhylmIm9535r93AE+RkYhKDQQ== X-Received: by 2002:a7b:cb4c:: with SMTP id v12mr29659257wmj.43.1593007498287; Wed, 24 Jun 2020 07:04:58 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 07/25] tests/vm: Added a new script for ubuntu.aarch64. Date: Wed, 24 Jun 2020 15:04:28 +0100 Message-Id: <20200624140446.15380-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x333.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-6-robert.foley@linaro.org> --- configure | 20 +++++++ tests/vm/Makefile.include | 11 ++++ tests/vm/aarch64vm.py | 106 ++++++++++++++++++++++++++++++++++++++ tests/vm/basevm.py | 12 +++++ tests/vm/ubuntu.aarch64 | 68 ++++++++++++++++++++++++ 5 files changed, 217 insertions(+) create mode 100644 tests/vm/aarch64vm.py create mode 100755 tests/vm/ubuntu.aarch64 diff --git a/configure b/configure index c1ec271931f..980a323ac7f 100755 --- a/configure +++ b/configure @@ -417,6 +417,7 @@ prefix=3D"/usr/local" mandir=3D"\${prefix}/share/man" datadir=3D"\${prefix}/share" firmwarepath=3D"\${prefix}/share/qemu-firmware" +efi_aarch64=3D"" qemu_docdir=3D"\${prefix}/share/doc/qemu" bindir=3D"\${prefix}/bin" libdir=3D"\${prefix}/lib" @@ -1107,6 +1108,8 @@ for opt do ;; --firmwarepath=3D*) firmwarepath=3D"$optarg" ;; + --efi-aarch64=3D*) efi_aarch64=3D"$optarg" + ;; --host=3D*|--build=3D*|\ --disable-dependency-tracking|\ --sbindir=3D*|--sharedstatedir=3D*|\ @@ -1777,6 +1780,7 @@ Advanced options (experts only): --sysconfdir=3DPATH install config in PATH$confsuffix --localstatedir=3DPATH install local state in PATH (set at runtime o= n win32) --firmwarepath=3DPATH search PATH for firmware files + --efi-aarch64=3DPATH PATH of efi file to use for aarch64 VMs. --with-confsuffix=3DSUFFIX suffix for QEMU data inside datadir/libdir/sy= sconfdir [$confsuffix] --with-pkgversion=3DVERS use specified string as sub-version of the pa= ckage --enable-debug enable common debug build options @@ -3593,6 +3597,20 @@ EOF fi fi =20 +############################################ +# efi-aarch64 probe +# Check for efi files needed by aarch64 VMs. +# By default we will use the efi included with QEMU. +# Allow user to override the path for efi also. +if ! test -f "$efi_aarch64"; then + if test -f $source_path/pc-bios/edk2-aarch64-code.fd.bz2; then + # valid after build + efi_aarch64=3D$PWD/pc-bios/edk2-aarch64-code.fd + else + efi_aarch64=3D"" + fi +fi + ########################################## # libcap-ng library probe if test "$cap_ng" !=3D "no" ; then @@ -6758,6 +6776,7 @@ if test "$docs" !=3D "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "efi_aarch64 $efi_aarch64" echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" !=3D "no" ; then @@ -7838,6 +7857,7 @@ echo "PYTHON=3D$python" >> $config_host_mak echo "SPHINX_BUILD=3D$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=3D$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=3D$genisoimage" >> $config_host_mak +echo "EFI_AARCH64=3D$efi_aarch64" >> $config_host_mak echo "PYTHON_YAML=3D$python_yaml" >> $config_host_mak echo "CC=3D$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index f6c3892bb28..4fa292765d5 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -5,6 +5,9 @@ IMAGES :=3D freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES +=3D ubuntu.i386 centos +ifneq ($(EFI_AARCH64),) +IMAGES +=3D ubuntu.aarch64 +endif endif =20 IMAGES_DIR :=3D $(HOME)/.cache/qemu-vm/images @@ -23,6 +26,11 @@ vm-help vm-test: ifneq ($(GENISOIMAGE),) @echo " vm-build-centos - Build QEMU in CentOS VM, with = Docker" @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" +ifneq ($(EFI_AARCH64),) + @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 V= M" +else + @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch= 64)" +endif else @echo " (install genisoimage to build centos/ubuntu images)" endif @@ -65,6 +73,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(V)$(DEBUG), --debug) \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$@" \ --force \ --build-image $@, \ @@ -80,6 +89,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -102,6 +112,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/aarch64vm.py b/tests/vm/aarch64vm.py new file mode 100644 index 00000000000..bb04cb19c91 --- /dev/null +++ b/tests/vm/aarch64vm.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# +# VM testing aarch64 library +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import os +import sys +import subprocess +import basevm +from qemu.accel import kvm_available + +# This is the config needed for current version of QEMU. +# This works for both kvm and tcg. +CURRENT_CONFIG =3D { + 'cpu' : "max", + 'machine' : "virt,gic-version=3Dmax", +} + +# The minimum minor version of QEMU we will support with aarch64 VMs is 3. +# QEMU versions less than 3 have various issues running these VMs. +QEMU_AARCH64_MIN_VERSION =3D 3 + +# The DEFAULT_CONFIG will default to a version of +# parameters that works for backwards compatibility. +DEFAULT_CONFIG =3D {'kvm' : {'cpu' : "host", + 'machine' : "virt,gic-version=3Dhost"}, + 'tcg' : {'cpu' : "cortex-a57", + 'machine' : "virt"}, +} + +def get_config_defaults(vmcls, default_config): + """Fetch the configuration defaults for this VM, + taking into consideration the defaults for + aarch64 first, followed by the defaults for this VM.""" + config =3D default_config + config.update(aarch_get_config_defaults(vmcls)) + return config + +def aarch_get_config_defaults(vmcls): + """Set the defaults for current version of QEMU.""" + config =3D CURRENT_CONFIG + args, argv =3D basevm.parse_args(vmcls) + qemu_path =3D basevm.get_qemu_path(vmcls.arch, args.build_path) + qemu_version =3D basevm.get_qemu_version(qemu_path) + if qemu_version < QEMU_AARCH64_MIN_VERSION: + error =3D "\nThis major version of QEMU {} is to old for aarch64 V= Ms.\n"\ + "The major version must be at least {}.\n"\ + "To continue with the current build of QEMU, "\ + "please restart with QEMU_LOCAL=3D1 .\n" + print(error.format(qemu_version, QEMU_AARCH64_MIN_VERSION)) + exit(1) + if qemu_version =3D=3D QEMU_AARCH64_MIN_VERSION: + # We have an older version of QEMU, + # set the config values for backwards compatibility. + if kvm_available('aarch64'): + config.update(DEFAULT_CONFIG['kvm']) + else: + config.update(DEFAULT_CONFIG['tcg']) + return config + +def create_flash_images(flash_dir=3D"./", efi_img=3D""): + """Creates the appropriate pflash files + for an aarch64 VM.""" + flash0_path =3D get_flash_path(flash_dir, "flash0") + flash1_path =3D get_flash_path(flash_dir, "flash1") + fd_null =3D open(os.devnull, 'w') + subprocess.check_call(["dd", "if=3D/dev/zero", "of=3D{}".format(flash0= _path), + "bs=3D1M", "count=3D64"], + stdout=3Dfd_null, stderr=3Dsubprocess.STDOUT) + # A reliable way to get the QEMU EFI image is via an installed package= or + # via the bios included with qemu. + if not os.path.exists(efi_img): + sys.stderr.write("*** efi argument is invalid ({})\n".format(efi_i= mg)) + sys.stderr.write("*** please check --efi-aarch64 argument or "\ + "install qemu-efi-aarch64 package\n") + exit(3) + subprocess.check_call(["dd", "if=3D{}".format(efi_img), + "of=3D{}".format(flash0_path), + "conv=3Dnotrunc"], + stdout=3Dfd_null, stderr=3Dsubprocess.STDOUT) + subprocess.check_call(["dd", "if=3D/dev/zero", + "of=3D{}".format(flash1_path), + "bs=3D1M", "count=3D64"], + stdout=3Dfd_null, stderr=3Dsubprocess.STDOUT) + fd_null.close() + +def get_pflash_args(flash_dir=3D"./"): + """Returns a string that can be used to + boot qemu using the appropriate pflash files + for aarch64.""" + flash0_path =3D get_flash_path(flash_dir, "flash0") + flash1_path =3D get_flash_path(flash_dir, "flash1") + pflash_args_str =3D "-drive file=3D{},format=3Draw,if=3Dpflash "\ + "-drive file=3D{},format=3Draw,if=3Dpflash" + pflash_args =3D pflash_args_str.format(flash0_path, flash1_path) + return pflash_args.split(" ") + +def get_flash_path(flash_dir, name): + return os.path.join(flash_dir, "{}.img".format(name)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index fa56fbbb4b6..5fd66f6b26a 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -92,6 +92,7 @@ class BaseVM(object): self._guest =3D None self._genisoimage =3D args.genisoimage self._build_path =3D args.build_path + self._efi_aarch64 =3D args.efi_aarch64 # Allow input config to override defaults. self._config =3D DEFAULT_CONFIG.copy() if config !=3D None: @@ -496,6 +497,14 @@ def get_qemu_path(arch, build_path=3DNone): qemu_path =3D "qemu-system-" + arch return qemu_path =20 +def get_qemu_version(qemu_path): + """Get the version number from the current QEMU, + and return the major number.""" + output =3D subprocess.check_output([qemu_path, '--version']) + version_line =3D output.decode("utf-8") + version_num =3D re.split(' |\(', version_line)[3].split('.')[0] + return int(version_num) + def parse_config(config, args): """ Parse yaml config and populate our config structure. The yaml config allows the user to override the @@ -573,6 +582,9 @@ def parse_args(vmcls): parser.add_option("--config", "-c", default=3DNone, help=3D"Provide config yaml for configuration. "\ "See config_example.yaml for example.") + parser.add_option("--efi-aarch64", + default=3D"/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", + help=3D"Path to efi image for aarch64 VMs.") parser.disable_interspersed_args() return parser.parse_args() =20 diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 new file mode 100755 index 00000000000..21d454c27f1 --- /dev/null +++ b/tests/vm/ubuntu.aarch64 @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 +# +# Ubuntu aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import sys +import basevm +import aarch64vm +import ubuntuvm + +DEFAULT_CONFIG =3D { + 'cpu' : "cortex-a57", + 'machine' : "virt,gic-version=3D3", + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y --arch-only qemu,"\ + "apt-get install -y libfdt-dev pkg-config language-pa= ck-en", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM + # especially using softmmu. + 'ssh_timeout' : 60, +} + +class UbuntuAarch64VM(ubuntuvm.UbuntuVM): + name =3D "ubuntu.aarch64" + arch =3D "aarch64" + image_name =3D "ubuntu-18.04-server-cloudimg-arm64.img" + image_link =3D "https://cloud-images.ubuntu.com/releases/18.04/release= /" + image_name + image_sha256=3D"0fdcba761965735a8a903d8b88df8e47f156f48715c00508e4315c= 506d7d3cb1" + BUILD_SCRIPT =3D """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=3D.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def boot(self, img, extra_args=3DNone): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args =3D aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args =3D default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to chang= e it. + extra_args.extend(["-smp", "8"]) + + # We have overridden boot() since aarch64 has additional parameter= s. + # Call down to the base class method. + super(UbuntuAarch64VM, self).boot(img, extra_args=3Dextra_args) + +if __name__ =3D=3D "__main__": + defaults =3D aarch64vm.get_config_defaults(UbuntuAarch64VM, DEFAULT_CO= NFIG) + sys.exit(basevm.main(UbuntuAarch64VM, defaults)) --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007657; cv=none; d=zohomail.com; s=zohoarc; b=JjhJQ9/CI45fYk1VfWJo/QNaUfhIxIhZT4mMhPM0+eKQSPiTucBYPemksw77NzB1WNSrfxppN4FiKrzJq9W6g4p7vH33Oi/aGc4qbWjXTu0vxDR6LBdueUCoGG70dYF1fP0vb80svN1h5y+dGBX1+E9vCBhljVRsn7kUD4n4JmU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007657; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kdMIZtPe+aZda1/mXQW7aQb8KseTM0tT+oUaOxEz8I4=; b=cd7jad1AqM+0b1d2UJNDEy3zCTDYNJagqlwC8vZ2gcGUCLItbDtQbT6kh+V9wh14CXduUpPrUv/jpfCBokcjFjq3HuNft57oXj9I4lvHXOUK82Bbtqlell+eC5u14vxtvRLmbz1JdjpkEUGSIvaF8xVX7ateJYH4uNxpmSqOVxk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007657351248.3181400355695; Wed, 24 Jun 2020 07:07:37 -0700 (PDT) Received: from localhost ([::1]:42156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo63r-0001vK-QJ for importer@patchew.org; Wed, 24 Jun 2020 10:07:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61Q-00051J-Hz for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:04 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:47049) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61N-0003Vu-9C for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:04 -0400 Received: by mail-wr1-x42c.google.com with SMTP id r12so2376922wrj.13 for ; Wed, 24 Jun 2020 07:05:00 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z1sm13677635wrh.14.2020.06.24.07.04.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:53 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 04B301FF96; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kdMIZtPe+aZda1/mXQW7aQb8KseTM0tT+oUaOxEz8I4=; b=MUJxn1sxXRAp4DhnvYyDZocNypsJUbI6Pp0bBQ4PYprWSSCQiUpWnpUclj80cUKLwp TjzCO0nh88sMGCxQ4+QHsgXMT+xkABjJkf+cVaqKpQEOw4H0KsjE+Bt5upJ7DpCkuD/G IwQb7u4TthDjDOPygwgyYE8zoWmhHHEQtG1HWrp5lH0pcMkU/Ek2aBf66lvc4eiGcN+4 ChOnEfv1wSe0Hrs8/awuS8KCaO7B0mzqwpAlkh3XyfgUOPq2ERrbAe5AIlqG89HRQieX 2E55R2GFsvuhOXOzUPGb0IIsDQsMkzTIMtZvQq4IID6MzQPcKQQ4ZvHyZQs/yfmX3TKG 8ljA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kdMIZtPe+aZda1/mXQW7aQb8KseTM0tT+oUaOxEz8I4=; b=QhPKVTclEaObuCw+O9Q+m4DwxZycINSKfDbtiyGKhTKKZZTV6IijxkDmXwmUt8mSAh ULir88rJHPmntm/ut2BV5ygP7imHlUQqPR3UBtKEQcbycJUM2GDqVrPghujoW6Pzw7YJ gsjzleO82gMC6ZsXwgAOL9x64XMGSTEBJOrts3wis9ExTHhWC/kCoHsudN43FyD5VmLR bkEbOCzI2lNl+Qb48bH1nVdlVvcoun7NEX7oVYsaZNMOwQB5Ceiv+NMNIhEqC9EXzRpD uAqVuFPbJRXw34GUR0VCjtSHqgjBdTN+BsmbzwXvLlf8lXx5+Juf93ypbcraj2GYM29V NIow== X-Gm-Message-State: AOAM533D54isK9lasU6Kg8o1dYilCTzx2RUUC1NZfJ8uUA2fDFWe9Zzn HTfRkgod2F5PuvsxSbTlvCYzdQ== X-Google-Smtp-Source: ABdhPJw7xJnqqpSQfZh5S9ZO1IZBMchobJRB8wC8vF4rn6QAE5h+XDM1G7IDFOYSGIP5k8Y906sojw== X-Received: by 2002:a5d:4603:: with SMTP id t3mr4807959wrq.38.1593007499536; Wed, 24 Jun 2020 07:04:59 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 08/25] tests/vm: Added a new script for centos.aarch64. Date: Wed, 24 Jun 2020 15:04:29 +0100 Message-Id: <20200624140446.15380-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley centos.aarch64 creates a CentOS 8 image. Also added a new kickstart script used to build the centos.aarch64 image. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Benn=C3=A9e Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-7-robert.foley@linaro.org> --- tests/vm/Makefile.include | 3 +- tests/vm/centos-8-aarch64.ks | 51 ++++++++ tests/vm/centos.aarch64 | 227 +++++++++++++++++++++++++++++++++++ 3 files changed, 280 insertions(+), 1 deletion(-) create mode 100644 tests/vm/centos-8-aarch64.ks create mode 100755 tests/vm/centos.aarch64 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 4fa292765d5..39f918a430a 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -6,7 +6,7 @@ IMAGES :=3D freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES +=3D ubuntu.i386 centos ifneq ($(EFI_AARCH64),) -IMAGES +=3D ubuntu.aarch64 +IMAGES +=3D ubuntu.aarch64 centos.aarch64 endif endif =20 @@ -28,6 +28,7 @@ ifneq ($(GENISOIMAGE),) @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" ifneq ($(EFI_AARCH64),) @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 V= M" + @echo " vm-build-centos.aarch64 - Build QEMU in CentOS aarch64 V= M" else @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch= 64)" endif diff --git a/tests/vm/centos-8-aarch64.ks b/tests/vm/centos-8-aarch64.ks new file mode 100644 index 00000000000..fd6ebe4d49e --- /dev/null +++ b/tests/vm/centos-8-aarch64.ks @@ -0,0 +1,51 @@ +# CentOS aarch64 image kickstart file. +# This file is used by the CentOS installer to +# script the generation of the image. +# +# Copyright 2020 Linaro +# +ignoredisk --only-use=3Dvda +# System bootloader configuration +bootloader --append=3D" crashkernel=3Dauto" --location=3Dmbr --boot-drive= =3Dvda +autopart --type=3Dplain +# Partition clearing information +clearpart --linux --initlabel --drives=3Dvda +# Use text mode install +text +repo --name=3D"AppStream" --baseurl=3Dfile:///run/install/repo/AppStream +# Use CDROM installation media +cdrom +# Keyboard layouts +keyboard --vckeymap=3Dus --xlayouts=3D'' +# System language +lang en_US.UTF-8 + +# Network information +network --bootproto=3Ddhcp --device=3Denp0s1 --onboot=3Doff --ipv6=3Dauto= --no-activate +network --hostname=3Dlocalhost.localdomain +# Run the Setup Agent on first boot +firstboot --enable +# Do not configure the X Window System +skipx +# System services +services --enabled=3D"chronyd" +# System timezone +timezone America/New_York --isUtc + +# Shutdown after installation is complete. +shutdown + +%packages +@^server-product-environment +kexec-tools + +%end + +%addon com_redhat_kdump --enable --reserve-mb=3D'auto' + +%end +%anaconda +pwpolicy root --minlen=3D6 --minquality=3D1 --notstrict --nochanges --note= mpty +pwpolicy user --minlen=3D6 --minquality=3D1 --notstrict --nochanges --empt= yok +pwpolicy luks --minlen=3D6 --minquality=3D1 --notstrict --nochanges --note= mpty +%end diff --git a/tests/vm/centos.aarch64 b/tests/vm/centos.aarch64 new file mode 100755 index 00000000000..d5232ecdb8c --- /dev/null +++ b/tests/vm/centos.aarch64 @@ -0,0 +1,227 @@ +#!/usr/bin/env python3 +# +# Centos aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.aarch64 +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import os +import sys +import subprocess +import basevm +import time +import traceback +import aarch64vm + +DEFAULT_CONFIG =3D { + 'cpu' : "max", + 'machine' : "virt,gic-version=3Dmax", + 'install_cmds' : "yum install -y make git python3 gcc gcc-c++ flex bis= on, "\ + "yum install -y glib2-devel pixman-devel zlib-devel, "\ + "yum install -y perl-Test-Harness, "\ + "alternatives --set python /usr/bin/python3, "\ + "sudo dnf config-manager "\ + "--add-repo=3Dhttps://download.docker.com/linux/centos/docker-ce.r= epo,"\ + "sudo dnf install --nobest -y docker-ce.aarch64,"\ + "systemctl enable docker", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM. + 'ssh_timeout' : 60, +} + +class CentosAarch64VM(basevm.BaseVM): + name =3D "centos.aarch64" + arch =3D "aarch64" + login_prompt =3D "localhost login:" + prompt =3D '[root@localhost ~]#' + image_name =3D "CentOS-8-aarch64-1905-dvd1.iso" + image_link =3D "http://mirrors.usc.edu/pub/linux/distributions/centos/= 8.0.1905/isos/aarch64/" + image_link +=3D image_name + BUILD_SCRIPT =3D """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=3D.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def set_key_perm(self): + """Set permissions properly on certain files to allow + ssh access.""" + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v /root/.ssh\n") + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v "\ + "/home/{}/.ssh\n".format(self._config["guest_user"])) + + def create_kickstart(self): + """Generate the kickstart file used to generate the centos image."= "" + # Start with the template for the kickstart. + ks_file =3D "../tests/vm/centos-8-aarch64.ks" + subprocess.check_call("cp {} ./ks.cfg".format(ks_file), shell=3DTr= ue) + # Append the ssh keys to the kickstart file + # as the post processing phase of installation. + with open("ks.cfg", "a") as f: + # Add in the root pw and guest user. + rootpw =3D "rootpw --plaintext {}\n" + f.write(rootpw.format(self._config["root_pass"])) + add_user =3D "user --groups=3Dwheel --name=3D{} "\ + "--password=3D{} --plaintext\n" + f.write(add_user.format(self._config["guest_user"], + self._config["guest_pass"])) + # Add the ssh keys. + f.write("%post --log=3D/root/ks-post.log\n") + f.write("mkdir -p /root/.ssh\n") + addkey =3D 'echo "{}" >> /root/.ssh/authorized_keys\n' + addkey_cmd =3D addkey.format(self._config["ssh_pub_key"]) + f.write(addkey_cmd) + f.write('mkdir -p /home/{}/.ssh\n'.format(self._config["guest_= user"])) + addkey =3D 'echo "{}" >> /home/{}/.ssh/authorized_keys\n' + addkey_cmd =3D addkey.format(self._config["ssh_pub_key"], + self._config["guest_user"]) + f.write(addkey_cmd) + f.write("%end\n") + # Take our kickstart file and create an .iso from it. + # The .iso will be provided to qemu as we boot + # from the install dvd. + # Anaconda will recognize the label "OEMDRV" and will + # start the automated installation. + gen_iso_img =3D 'genisoimage -output ks.iso -volid "OEMDRV" ks.cfg' + subprocess.check_call(gen_iso_img, shell=3DTrue) + + def wait_for_shutdown(self): + """We wait for qemu to shutdown the VM and exit. + While this happens we display the console view + for easier debugging.""" + # The image creation is essentially done, + # so whether or not the wait is successful we want to + # wait for qemu to exit (the self.wait()) before we return. + try: + self.console_wait("reboot: Power down") + except Exception as e: + sys.stderr.write("Exception hit\n") + if isinstance(e, SystemExit) and e.code =3D=3D 0: + return 0 + traceback.print_exc() + finally: + self.wait() + + def build_base_image(self, dest_img): + """Run through the centos installer to create + a base image with name dest_img.""" + # We create the temp image, and only rename + # to destination when we are done. + img =3D dest_img + ".tmp" + # Create an empty image. + # We will provide this as the install destination. + qemu_img_create =3D "qemu-img create {} 50G".format(img) + subprocess.check_call(qemu_img_create, shell=3DTrue) + + # Create our kickstart file to be fed to the installer. + self.create_kickstart() + # Boot the install dvd with the params as our ks.iso + os_img =3D self._download_with_cache(self.image_link) + dvd_iso =3D "centos-8-dvd.iso" + subprocess.check_call(["cp", "-f", os_img, dvd_iso]) + extra_args =3D "-cdrom ks.iso" + extra_args +=3D " -drive file=3D{},if=3Dnone,id=3Ddrive1,cache=3Dw= riteback" + extra_args +=3D " -device virtio-blk,drive=3Ddrive1,bootindex=3D1" + extra_args =3D extra_args.format(dvd_iso).split(" ") + self.boot(img, extra_args=3Dextra_args) + self.console_wait_send("change the selection", "\n") + # We seem to need to hit esc (chr(27)) twice to abort the + # media check, which takes a long time. + # Waiting a bit seems to be more reliable before hitting esc. + self.console_wait("Checking") + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + print("Found Checking") + # Give sufficient time for the installer to create the image. + self.console_init(timeout=3D7200) + self.wait_for_shutdown() + os.rename(img, dest_img) + print("Done with base image build: {}".format(dest_img)) + + def check_create_base_img(self, img_base, img_dest): + """Create a base image using the installer. + We will use the base image if it exists. + This helps cut down on install time in case we + need to restart image creation, + since the base image creation can take a long time.""" + if not os.path.exists(img_base): + print("Generate new base image: {}".format(img_base)) + self.build_base_image(img_base); + else: + print("Use existing base image: {}".format(img_base)) + # Save a copy of the base image and copy it to dest. + # which we will use going forward. + subprocess.check_call(["cp", img_base, img_dest]) + + def boot(self, img, extra_args=3DNone): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args =3D aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args =3D default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to chang= e it. + extra_args.extend(["-smp", "8"]) + # We have overridden boot() since aarch64 has additional parameter= s. + # Call down to the base class method. + super(CentosAarch64VM, self).boot(img, extra_args=3Dextra_args) + + def build_image(self, img): + img_tmp =3D img + ".tmp" + self.check_create_base_img(img + ".base", img_tmp) + + # Boot the new image for the first time to finish installation. + self.boot(img_tmp) + self.console_init() + self.console_wait_send(self.login_prompt, "root\n") + self.console_wait_send("Password:", + "{}\n".format(self._config["root_pass"])) + + self.set_key_perm() + self.console_wait_send(self.prompt, "rpm -q centos-release\n") + enable_adapter =3D "sed -i 's/ONBOOT=3Dno/ONBOOT=3Dyes/g'" \ + " /etc/sysconfig/network-scripts/ifcfg-enp0s1\n" + self.console_wait_send(self.prompt, enable_adapter) + self.console_wait_send(self.prompt, "ifup enp0s1\n") + self.console_wait_send(self.prompt, + 'echo "qemu ALL=3D(ALL) NOPASSWD:ALL" | '\ + 'sudo tee /etc/sudoers.d/qemu\n') + self.console_wait(self.prompt) + + # Rest of the commands we issue through ssh. + self.wait_ssh(wait_root=3DTrue) + + # If the user chooses *not* to do the second phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] !=3D "": + install_cmds =3D self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.ssh_root("poweroff") + self.wait_for_shutdown() + os.rename(img_tmp, img) + print("image creation complete: {}".format(img)) + return 0 + +if __name__ =3D=3D "__main__": + defaults =3D aarch64vm.get_config_defaults(CentosAarch64VM, DEFAULT_CO= NFIG) + sys.exit(basevm.main(CentosAarch64VM, defaults)) --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008015; cv=none; d=zohomail.com; s=zohoarc; b=ZRbckCrcOBqmwhC8iAnezSIfn8I5dMJhI7QI+2m6PABQQ52e8GL9lDbhWlkbAjNmI8NthoYMuYpG4ND6Yev/4i6YmmXpFYifspfzQ2FusQAHCzU9bifw67Ver//4itylj+j1y+eWD/z2OmYhKlnmCPfb8Z+sOdjoJH0h4y1DepQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008015; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qDw5tmNjtaEA2MrxzXTWX9WRb/d3hDPxln94PjTXX2k=; b=TP1ZSznlxy3oqsh6YIV9gwBFWaCRBMZ9MuJEJr7GkzBLGBVJ3P6OUwAg6dkrOiplPJ760jStDDM9+BJ5v6Jh33/NTYes/zlBNNHLxF4PKALL5DNVulJgGum4nK7kbiWwvh2KZV+NmBuv3177B8shzJ5XzJE3E6v38vnGJWgZ9R8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159300801589891.11071842948104; Wed, 24 Jun 2020 07:13:35 -0700 (PDT) Received: from localhost ([::1]:36172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo69e-0004Wi-Hg for importer@patchew.org; Wed, 24 Jun 2020 10:13:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59276) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61S-00054f-Dd for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:06 -0400 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:35518) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61O-0003W7-Gr for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:06 -0400 Received: by mail-wm1-x344.google.com with SMTP id l2so1173633wmf.0 for ; Wed, 24 Jun 2020 07:05:01 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id v7sm27540685wrp.45.2020.06.24.07.04.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:54 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 23DC41FF98; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qDw5tmNjtaEA2MrxzXTWX9WRb/d3hDPxln94PjTXX2k=; b=GaytSj8f3EocCFCEFEosAnZX2HysoeIThJ3E1560NouuX2QkT/59j+q2g/YsfNlZ5b nzyJ7yJiao3+8aNNqVOKmP6aKIp9Ni8aVPLWKzkDRATfZUxwilKtNjFMX0cJNDIDsvw2 njzv7z8N/f8rCNqlnupKX/WyAheC2Qw9TRxrqpvdMw7UTOLxo3UO/Uw40cTomTwXvbKE vCMOx6crgo73BKPxj19mJ1cQ32YQagGn33JwXPVsYgPpbaj5OqMDmfkx0W9rCrn9X0rN HSWAgbc8qZoNBQq8ofSbzx4EaDhIK3YIOElbbuzNrRfUWLIN6BNyJiJQh5zTAUioUhql ug+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qDw5tmNjtaEA2MrxzXTWX9WRb/d3hDPxln94PjTXX2k=; b=kPAkK9XL2ZgZ4As3UoRAW4aaXmyoxVxp2MWbdahc1H+jVrwi8oSxuFhLxHI+bC3KQV 3UnoQt6CVl1RGYRNdnvWUTn9RI+7clCy/3X0f4+BggEc1Bq/NBm+Ms6Ng4e9tophOA+V r0BT0PkGhKLzDmKzk1nb21v+5FX+n9HT0W5CW2aiGaRL2D+EEUOuxTaBUTX9+zsvT8sl GbaTeP6nGuFQkvXGxEU6k9VtiPCGUn5qsOOBMfwegXVUkn1YqMO0sp2q9UyF8Niggq3R banWJt6kx0aAq6miEleBXFBrht7fJWA0JpUS4I/hx0VSrl5d2dUhcUOajh77u/UzPSBY ureA== X-Gm-Message-State: AOAM531i4Ja/qPBokHU25V/3B0So5wPVfFUugxMg56aEOdUv8N3RVDXH aCQ43n3KKFNTq3CjYzXqDhTgQA== X-Google-Smtp-Source: ABdhPJyAnFYuZ+jN3YRboA3s2pmxeXXSEF96fUNKfsIUN6wwyy7UdPleRRirLlpe1N3NgLTK6hmofw== X-Received: by 2002:a1c:2146:: with SMTP id h67mr27970796wmh.46.1593007500867; Wed, 24 Jun 2020 07:05:00 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 09/25] tests/vm: change scripts to use self._config Date: Wed, 24 Jun 2020 15:04:30 +0100 Message-Id: <20200624140446.15380-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::344; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x344.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley This change converts existing scripts to using for example self.ROOT_PASS, to self._config['root_pass']. We made similar changes for GUEST_USER, and GUEST_PASS. This allows us also to remove the change in basevm.py, which adds __getattr__ for backwards compatibility. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-8-robert.foley@linaro.org> --- tests/vm/basevm.py | 11 ++--------- tests/vm/fedora | 17 +++++++++-------- tests/vm/freebsd | 16 ++++++++-------- tests/vm/netbsd | 19 ++++++++++--------- tests/vm/openbsd | 17 +++++++++-------- 5 files changed, 38 insertions(+), 42 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 5fd66f6b26a..f716798b405 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -183,13 +183,6 @@ class BaseVM(object): self.console_init(timeout=3Dtimeout) self.console_wait(wait_string) =20 - def __getattr__(self, name): - # Support direct access to config by key. - # for example, access self._config['cpu'] by self.cpu - if name.lower() in self._config.keys(): - return self._config[name.lower()] - return object.__getattribute__(self, name) - def _download_with_cache(self, url, sha256sum=3DNone, sha512sum=3DNone= ): def check_sha256sum(fname): if not sha256sum: @@ -239,13 +232,13 @@ class BaseVM(object): return r =20 def ssh(self, *cmd): - return self._ssh_do(self.GUEST_USER, cmd, False) + return self._ssh_do(self._config["guest_user"], cmd, False) =20 def ssh_root(self, *cmd): return self._ssh_do("root", cmd, False) =20 def ssh_check(self, *cmd): - self._ssh_do(self.GUEST_USER, cmd, True) + self._ssh_do(self._config["guest_user"], cmd, True) =20 def ssh_root_check(self, *cmd): self._ssh_do("root", cmd, True) diff --git a/tests/vm/fedora b/tests/vm/fedora index a9195670f4b..b2b478fdbca 100755 --- a/tests/vm/fedora +++ b/tests/vm/fedora @@ -108,20 +108,20 @@ class FedoraVM(basevm.BaseVM): =20 self.console_wait_send("7) [!] Root password", "7\n") self.console_wait("Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) =20 self.console_wait_send("8) [ ] User creation", "8\n") self.console_wait_send("1) [ ] Create user", "1\n") self.console_wait_send("3) User name", "3\n") - self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER) + self.console_wait_send("ENTER:", "%s\n" % self._config["guest_user= "]) self.console_wait_send("4) [ ] Use password", "4\n") self.console_wait_send("5) Password", "5\n") self.console_wait("Password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("7) Groups", "c\n") =20 while True: @@ -139,7 +139,7 @@ class FedoraVM(basevm.BaseVM): if good: break time.sleep(10) - self.console_send("r\n" % self.GUEST_PASS) + self.console_send("r\n" % self._config["guest_pass"]) =20 self.console_wait_send("'b' to begin install", "b\n") =20 @@ -150,12 +150,13 @@ class FedoraVM(basevm.BaseVM): =20 # setup qemu user prompt =3D " ~]$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") =20 # setup root user prompt =3D " ~]#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) =20 # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/freebsd b/tests/vm/freebsd index f87db2b126e..29252fa4a64 100755 --- a/tests/vm/freebsd +++ b/tests/vm/freebsd @@ -113,9 +113,9 @@ class FreeBSDVM(basevm.BaseVM): =20 # post-install configuration self.console_wait("New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) =20 self.console_wait_send("Network Configuration", "\n") self.console_wait_send("IPv4", "y") @@ -134,9 +134,9 @@ class FreeBSDVM(basevm.BaseVM): # qemu user self.console_wait_send("Add User Accounts", "y") self.console_wait("Username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait_send("Uid", "\n") self.console_wait_send("Login group", "\n") self.console_wait_send("Login group", "\n") @@ -148,9 +148,9 @@ class FreeBSDVM(basevm.BaseVM): self.console_wait_send("Use an empty password", "\n") self.console_wait_send("Use a random password", "\n") self.console_wait("Enter password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Enter password again:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Lock out", "\n") self.console_wait_send("OK", "yes\n") self.console_wait_send("Add another user", "no\n") @@ -164,12 +164,12 @@ class FreeBSDVM(basevm.BaseVM): =20 # setup qemu user prompt =3D "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], self._co= nfig["guest_pass"]) self.console_wait_send(prompt, "exit\n") =20 # setup root user prompt =3D "root@freebsd:~ #" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) =20 # setup serial console diff --git a/tests/vm/netbsd b/tests/vm/netbsd index cdac502dad8..dc0e45c1d45 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -120,24 +120,24 @@ class NetBSDVM(basevm.BaseVM): self.console_wait_send("d: Change root password", "d\n") self.console_wait_send("a: Yes", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) =20 self.console_wait_send("o: Add a user", "o\n") self.console_wait("username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("to group wheel") self.console_wait_send("a: Yes", "a\n") self.console_wait_send("a: /bin/sh", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) =20 self.console_wait_send("a: Configure network", "a\n") self.console_wait_send("a: vioif0", "a\n") @@ -170,12 +170,13 @@ class NetBSDVM(basevm.BaseVM): =20 # setup qemu user prompt =3D "localhost$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") =20 # setup root user prompt =3D "localhost#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) =20 # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/openbsd b/tests/vm/openbsd index 13e7f9a6d56..dfe633e4532 100755 --- a/tests/vm/openbsd +++ b/tests/vm/openbsd @@ -98,9 +98,9 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which network interface", "done\n") self.console_wait_send("DNS domain name", "localnet\n") self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Start sshd(8)", "yes\n") self.console_wait_send("X Window System", "\n") self.console_wait_send("xenodm", "\n") @@ -108,13 +108,13 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which speed", "\n") =20 self.console_wait("Setup a user") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) =20 self.console_wait_send("Allow root ssh login", "yes\n") self.console_wait_send("timezone", "UTC\n") @@ -135,12 +135,13 @@ class OpenBSDVM(basevm.BaseVM): =20 # setup qemu user prompt =3D "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") =20 # setup root user prompt =3D "openbsd#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) =20 # setup virtio-blk #1 (tarfile) --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007738; cv=none; d=zohomail.com; s=zohoarc; b=NuCVuSl2EIfQKyU2RaDYvOTpiZmveLBh0C5dmHGkjDgFGKuGMvMYj/OfLjlWSVhVaEHcCvXPb4CTlQSo5j5psBgE/UHTagKD/Gi2eDgLDizIpjM4x1hv26iu/E+PUs9Z56dzxyeTWZX6uDiVP39UbRUuyzWytOFlfPSLrP9krc0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007738; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SfCKKjXRpSzktD2zHuCdVYnzBzWod3odSIiX8dUA3RA=; b=jkJm4WCrPNJcnfTQo8JsUH4CT7ZB4NjeonCrL4ykJEb7tf9PtSGeQymh/fa5zSQlaJzEdjjEiG+iW0C1zbWm9hxsGS5oHvj0BBHzHe5JFH9l+1b8K54Uu/MpdVoBnxJfXFI3WoXs0O/pQetqoCVv7CgUMiOj8CYDUqg57pfgCMY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159300773840911.744981774935923; Wed, 24 Jun 2020 07:08:58 -0700 (PDT) Received: from localhost ([::1]:50528 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo65B-0005YN-9n for importer@patchew.org; Wed, 24 Jun 2020 10:08:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61S-00053x-5y for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:06 -0400 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:37766) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61P-0003WQ-Nd for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:05 -0400 Received: by mail-wr1-x42b.google.com with SMTP id a6so2429962wrm.4 for ; Wed, 24 Jun 2020 07:05:03 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id l132sm7688044wmf.6.2020.06.24.07.04.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:54 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 3DE791FF99; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SfCKKjXRpSzktD2zHuCdVYnzBzWod3odSIiX8dUA3RA=; b=UM4qUG57JiusQsCdTNT8H0O6h2xjNdxLbCahDXyz5GfqlW4t7ReyWdji7EEWjMDo2l JR6S0lsFcuD6zwzY/wSFXTZd6IAHZ+EXSNLHz/KVeF+D+fEWuuuv86dCB3BA0p0tNA91 42uNASYp6e9XvHeX2Pm+TI6o6/Tln3l0jvL+hxb8fAvfI8qG+fUXrjhUq/BsCpoYUomW dcVKJDfQ5lmPDXy80Xxa6CRTVDcINa+TTrHlYMImtAJjWzH6yIq4c2TsdyW3iom+PFFk 17bgQWv2UTDyIOXuH+EBzZWQY9C+heeAqfEWh6ViiauyMhe7Hv4Ol5QGufxabB4v/EqH +78w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SfCKKjXRpSzktD2zHuCdVYnzBzWod3odSIiX8dUA3RA=; b=JMrU5Az78C33kU6jkvSKtgEy8ciRBzvfHQ0YXEmM4MRtLfIy7aQumwKOhuRw0KPq0D uAQE2N+lStRH5hCliWOGpz0qrgPl21dGlDMJMCr4kJWipb/Xjw6phFMFTxhZPSL4bcPd 37HQ66VMM3DKe1e1CDsjiXEu5VW+bumhUxnUhJ7UmjkBdDTywTLAifZ83iFRSh8b1GB4 rpE89ygZvJC3QZ4oMNvmUFkznNJBTT1ee9fPXZT3ONvEgcfKyBQnkrQkRoEAcbB+MtIp iuEmZDNQ2iiOdduZ118gpLDiadUOny4nAhFvn26GJQkLWIXERLU4T12Sw7lNn5VpTKl4 OVdw== X-Gm-Message-State: AOAM531CHdJqY2olSqMwGofnLMVVCcf3Ta7bG2Eo/fR6QlRO7lybsmdg IDNpmQkcghRuGzjl8vgp6fqkxw== X-Google-Smtp-Source: ABdhPJxXvIysnc0ZgkD6ZJwI5+hg3d3ecTBVljuu2Am7cb56YRgHy2Cz2GjyNoAUSglLRXcBl41rzw== X-Received: by 2002:a5d:6052:: with SMTP id j18mr30206248wrt.387.1593007502201; Wed, 24 Jun 2020 07:05:02 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 10/25] python/qemu: Add ConsoleSocket for optional use in QEMUMachine Date: Wed, 24 Jun 2020 15:04:31 +0100 Message-Id: <20200624140446.15380-11-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Cleber Rosa , Peter Puhov , aurelien@aurel32.net, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley We add the ConsoleSocket object, which has a socket interface and which will consume all arriving characters on the socket, placing them into an in memory buffer. This will also provide those chars via recv() as would a regular socket. ConsoleSocket also has the option of dumping the console bytes to a log file. We also give QEMUMachine the option of using ConsoleSocket to drain and to use for logging console to a file. By default QEMUMachine does not use ConsoleSocket. This is added in preparation for use by basevm.py in a later commit. This is a workaround we found was needed for basevm.py since there is a known issue where QEMU will hang waiting for console characters to be consumed. Cc: Eduardo Habkost Cc: Cleber Rosa Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Benn=C3=A9e Tested-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-9-robert.foley@linaro.org> --- python/qemu/console_socket.py | 110 ++++++++++++++++++++++++++++++++++ python/qemu/machine.py | 23 +++++-- 2 files changed, 129 insertions(+), 4 deletions(-) create mode 100644 python/qemu/console_socket.py diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py new file mode 100644 index 00000000000..830cb7c6282 --- /dev/null +++ b/python/qemu/console_socket.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +# +# This python module implements a ConsoleSocket object which is +# designed always drain the socket itself, and place +# the bytes into a in memory buffer for later processing. +# +# Optionally a file path can be passed in and we will also +# dump the characters to this file for debug. +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import asyncore +import socket +import threading +import io +import os +import sys +from collections import deque +import time +import traceback + +class ConsoleSocket(asyncore.dispatcher): + + def __init__(self, address, file=3DNone): + self._recv_timeout_sec =3D 300 + self._buffer =3D deque() + self._asyncore_thread =3D None + self._sock =3D socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self._sock.connect(address) + self._logfile =3D None + if file: + self._logfile =3D open(file, "w") + asyncore.dispatcher.__init__(self, sock=3Dself._sock) + self._open =3D True + self._thread_start() + + def _thread_start(self): + """Kick off a thread to wait on the asyncore.loop""" + if self._asyncore_thread is not None: + return + self._asyncore_thread =3D threading.Thread(target=3Dasyncore.loop, + kwargs=3D{'timeout':1}) + self._asyncore_thread.daemon =3D True + self._asyncore_thread.start() + + def handle_close(self): + """redirect close to base class""" + # Call the base class close, but not self.close() since + # handle_close() occurs in the context of the thread which + # self.close() attempts to join. + asyncore.dispatcher.close(self) + + def close(self): + """Close the base object and wait for the thread to terminate""" + if self._open: + self._open =3D False + asyncore.dispatcher.close(self) + if self._asyncore_thread is not None: + thread, self._asyncore_thread =3D self._asyncore_thread, N= one + thread.join() + if self._logfile: + self._logfile.close() + self._logfile =3D None + + def handle_read(self): + """process arriving characters into in memory _buffer""" + try: + data =3D asyncore.dispatcher.recv(self, 1) + # latin1 is needed since there are some chars + # we are receiving that cannot be encoded to utf-8 + # such as 0xe2, 0x80, 0xA6. + string =3D data.decode("latin1") + except: + print("Exception seen.") + traceback.print_exc() + return + if self._logfile: + self._logfile.write("{}".format(string)) + self._logfile.flush() + for c in string: + self._buffer.extend(c) + + def recv(self, n=3D1, sleep_delay_s=3D0.1): + """Return chars from in memory buffer""" + start_time =3D time.time() + while len(self._buffer) < n: + time.sleep(sleep_delay_s) + elapsed_sec =3D time.time() - start_time + if elapsed_sec > self._recv_timeout_sec: + raise socket.timeout + chars =3D ''.join([self._buffer.popleft() for i in range(n)]) + # We choose to use latin1 to remain consistent with + # handle_read() and give back the same data as the user would + # receive if they were reading directly from the + # socket w/o our intervention. + return chars.encode("latin1") + + def set_blocking(self): + """Maintain compatibility with socket API""" + pass + + def settimeout(self, seconds): + """Set current timeout on recv""" + self._recv_timeout_sec =3D seconds diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 041c615052e..c25f0b42cf6 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -26,6 +26,7 @@ import socket import tempfile from typing import Optional, Type from types import TracebackType +from qemu.console_socket import ConsoleSocket =20 from . import qmp =20 @@ -75,7 +76,8 @@ class QEMUMachine: =20 def __init__(self, binary, args=3DNone, wrapper=3DNone, name=3DNone, test_dir=3D"/var/tmp", monitor_address=3DNone, - socket_scm_helper=3DNone, sock_dir=3DNone): + socket_scm_helper=3DNone, sock_dir=3DNone, + drain_console=3DFalse, console_log=3DNone): ''' Initialize a QEMUMachine =20 @@ -86,6 +88,9 @@ class QEMUMachine: @param test_dir: where to create socket and log file @param monitor_address: address for QMP monitor @param socket_scm_helper: helper program, required for send_fd_scm= () + @param sock_dir: where to create socket (overrides test_dir for so= ck) + @param console_log: (optional) path to console log file + @param drain_console: (optional) True to drain console socket to b= uffer @note: Qemu process is not started until launch() is used. ''' if args is None: @@ -122,6 +127,12 @@ class QEMUMachine: self._console_address =3D None self._console_socket =3D None self._remove_files =3D [] + self._console_log_path =3D console_log + if self._console_log_path: + # In order to log the console, buffering needs to be enabled. + self._drain_console =3D True + else: + self._drain_console =3D drain_console =20 def __enter__(self): return self @@ -580,7 +591,11 @@ class QEMUMachine: Returns a socket connected to the console """ if self._console_socket is None: - self._console_socket =3D socket.socket(socket.AF_UNIX, - socket.SOCK_STREAM) - self._console_socket.connect(self._console_address) + if self._drain_console: + self._console_socket =3D ConsoleSocket(self._console_addre= ss, + file=3Dself._console_l= og_path) + else: + self._console_socket =3D socket.socket(socket.AF_UNIX, + socket.SOCK_STREAM) + self._console_socket.connect(self._console_address) return self._console_socket --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007744; cv=none; d=zohomail.com; s=zohoarc; b=eEKEvWQ8sxmnbismb6ekZOy9dU+CzoU7838bXNWVBSzZkHd9nrHqwHu7xKDwr0Gd4kHggjUeqdi4nxKFC4pEcfrYPVp2PaW1d4YznNfpDqT1ZY97av/NZDs7G4WSekwVBaH2t0INCoAt0J0sOq7U7eaiuCn1wOgo1zK9heGgJoY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007744; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5xSn2L6kzBKcKjTR8zPwxbT757SBhdGuGZ/3k/jBdz8=; b=lhfrjufSYsZvE8ll5r5dAVpqp89r7z0HTamlfSIq+NcaHO9Pk5PLUueyh5xYCcELaljUKdCG0EuM1PNX6BIZN5CcetkHBhFfXWT9sHlZ/uGGxPCh9kLP5XWhSd5cqDUE6Oz4AXLu/HYcsB+Ra1J1yeAvEhMhIEhUQyL4MiCZIVs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007744843978.5884206676358; Wed, 24 Jun 2020 07:09:04 -0700 (PDT) Received: from localhost ([::1]:50936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo65H-0005iN-Di for importer@patchew.org; Wed, 24 Jun 2020 10:09:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61N-0004vS-8D for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:01 -0400 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:55384) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61K-0003VM-9j for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:00 -0400 Received: by mail-wm1-x343.google.com with SMTP id g75so2435444wme.5 for ; Wed, 24 Jun 2020 07:04:57 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n5sm3827122wmi.34.2020.06.24.07.04.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 568931FF9A; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5xSn2L6kzBKcKjTR8zPwxbT757SBhdGuGZ/3k/jBdz8=; b=stLkwgi/ueBYCoZNMDFasbFrfEB3RKeBudIM3YKlcDrgHLPWVWK5ouOMmNfFSV48ks BtxR1AZMtI+btE5adAcEOuOi0b8DTaTc77DS1QCuLlPU4Wd2Zs82XGRynnbNdC5pnK5w XXLmc/W+8VhXcPlsGKu+l6cyNVDFqe/py6JwzEkrVQKo8IUup2p4/0mACMFtZldUnj1r 9uCKIXIe8WldmyPrQ84vipeC5PIOf1HusiRaC7r5ZZSfdbhj9KGbJ5DY8RgV9fRK3va9 /fmx3iQRRHevyNzzkkixTIyKqwgAUOkhzMjmzwu7VL5llfOu7VW7Y9NHvXJ6sRHYUVyk OKlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5xSn2L6kzBKcKjTR8zPwxbT757SBhdGuGZ/3k/jBdz8=; b=GgXD8M8fALajjj2Z7Jh1UMqy0ceRVlqb6Akqr5R5+EoxGlFmlIXMa9X2fHljYFkpCn 132ezZodOEzFqgYdGg18azhBtitv02UzggXQ6mHnYqY9S9BLFqfq5DXRkSfm6JT3dhUA nhuysq+6WBF/i46JcPwJfsJHmLJnEBlLNO+6N1OJ4S25jN1LauziHXfHgOM+HF7+3hTo bRA35SN5D3CThV9zuQYT//fnFuXafUsQkxqTB6lsMFvSG8exURjPZqiD8ZRnAKNgGdC5 Y9x3RjrjaF8b6pCR0c/+OXoVIDu7kWPIWDv5tiF6WpEbisRK547vtM7bbFqT9VOQZZd1 X4zA== X-Gm-Message-State: AOAM532wUJLQ0kaKDh1+Ib/FVu5+gpH9dVwOKmMh+Jn+jit0ohIimGAa 7Xt1GWO8DOWptOPi+khJyqAhcw== X-Google-Smtp-Source: ABdhPJxW3eE7QnWJIGbC577I2DcCZbNkmUNL4erxPnYOsjK6nrEezWs5YSMysaP+ErivvyDyYjHMxg== X-Received: by 2002:a1c:e209:: with SMTP id z9mr5757339wmg.153.1593007496890; Wed, 24 Jun 2020 07:04:56 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 11/25] tests/vm: Add workaround to consume console Date: Wed, 24 Jun 2020 15:04:32 +0100 Message-Id: <20200624140446.15380-12-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::343; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x343.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Robert Foley This adds support to basevm.py so that we always drain the console chars. This makes use of support added in an earlier commit that allows QEMUMachine to use the ConsoleSocket. This is a workaround we found was needed since there is a known issue where QEMU will hang waiting for console characters to be consumed. We also added the option of logging the console to a file. LOG_CONSOLE=3D1 will now log the output to a file. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Benn=C3=A9e Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200601211421.1277-10-robert.foley@linaro.org> --- tests/vm/Makefile.include | 4 ++++ tests/vm/basevm.py | 21 +++++++++++++++++++-- 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 39f918a430a..f21948c46a5 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -49,6 +49,7 @@ endif @echo ' EXTRA_CONFIGURE_OPTS=3D"..."' @echo " J=3D[0..9]* - Override the -jN parameter for make= commands" @echo " DEBUG=3D1 - Enable verbose output on host and i= nteractive debugging" + @echo " LOG_CONSOLE=3D1 - Log console to file in: ~/.cache/qe= mu-vm " @echo " V=3D1 - Enable verbose ouput on host and guest commands" @echo " QEMU_LOCAL=3D1 - Use QEMU binary local to this= build." @echo " QEMU=3D/path/to/qemu - Change path to QEMU binary" @@ -75,6 +76,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$@" \ --force \ --build-image $@, \ @@ -91,6 +93,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -114,6 +117,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index f716798b405..04d083409a5 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -117,6 +117,11 @@ class BaseVM(object): "w").write(self._config['ssh_pub_key']) =20 self.debug =3D args.debug + self._console_log_path =3D None + if args.log_console: + self._console_log_path =3D \ + os.path.join(os.path.expanduser("~/.cache/qemu-vm= "), + "{}.install.log".format(self.name)) self._stderr =3D sys.stderr self._devnull =3D open(os.devnull, "w") if self.debug: @@ -271,7 +276,13 @@ class BaseVM(object): args +=3D self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_path =3D get_qemu_path(self.arch, self._build_path) - guest =3D QEMUMachine(binary=3Dqemu_path, args=3Dargs) + + # Since console_log_path is only set when the user provides the + # log_console option, we will set drain_console=3DTrue so the + # console is always drained. + guest =3D QEMUMachine(binary=3Dqemu_path, args=3Dargs, + console_log=3Dself._console_log_path, + drain_console=3DTrue) guest.set_machine(self._config['machine']) guest.set_console() try: @@ -285,6 +296,8 @@ class BaseVM(object): raise atexit.register(self.shutdown) self._guest =3D guest + # Init console so we can start consuming the chars. + self.console_init() usernet_info =3D guest.qmp("human-monitor-command", command_line=3D"info usernet") self.ssh_port =3D None @@ -296,7 +309,9 @@ class BaseVM(object): raise Exception("Cannot find ssh port from 'info usernet':\n%s= " % \ usernet_info) =20 - def console_init(self, timeout =3D 120): + def console_init(self, timeout =3D None): + if timeout =3D=3D None: + timeout =3D self.socket_timeout vm =3D self._guest vm.console_socket.settimeout(timeout) self.console_raw_path =3D os.path.join(vm._temp_dir, @@ -578,6 +593,8 @@ def parse_args(vmcls): parser.add_option("--efi-aarch64", default=3D"/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", help=3D"Path to efi image for aarch64 VMs.") + parser.add_option("--log-console", action=3D"store_true", + help=3D"Log console to file.") parser.disable_interspersed_args() return parser.parse_args() =20 --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007664; cv=none; d=zohomail.com; s=zohoarc; b=gMdP8EryPGzeUH2GI6Pr60Vx5pW3ybekh0SRrT9opjFuVx9XJP4AdP4cXdzQg1DFi+s7ZsNW3AZhzTZNlHRKELpkUttrj3fVTVrJgjU2nd8b/wZ5ns0I76ySQ7MnX4SEXJMpHVVid6xwijHllDa4K+02XERs3SxMTS/WsaYcCBY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007664; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MIJDY0wCVkf83j0A2folG6JUn+sBiPxS+WIpYPexzXA=; b=gGfjUXZnoCudehS5UrHg31tirY/r6t9gqvsLmWf0htLzufVgmWN58KoI+zX4gnbYU+feSP4fncwl0Ylql5XH6aHd+jyVLQgg8YGobe/FI0kC12AU7pUSdRMXY/ThOVZvpZOI0HDouOYMSefzJTp6/Qu5VXD1uv30oNxoT8Hd8Zg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007664712859.5115583353635; Wed, 24 Jun 2020 07:07:44 -0700 (PDT) Received: from localhost ([::1]:42892 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo63y-0002DJ-Ro for importer@patchew.org; Wed, 24 Jun 2020 10:07:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59290) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61U-000582-2o for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:08 -0400 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:46114) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61R-0003Wo-RO for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:07 -0400 Received: by mail-wr1-x443.google.com with SMTP id r12so2377264wrj.13 for ; Wed, 24 Jun 2020 07:05:05 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n5sm3827123wmi.34.2020.06.24.07.04.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 6D3061FF9B; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MIJDY0wCVkf83j0A2folG6JUn+sBiPxS+WIpYPexzXA=; b=amljOvBKYb19XKL33ec5BrEZxXJlWJo9bRCzIKec77yGf0buKkslUQThM9fO91t4iF 43aL3IhhX9QgVp+Y4RBLFKpfTGeTD04zBiKemPeRlD4YFBMkHx7wVEjDuLG35La+Lip1 74DKs6TNAbU61X2AffvS2oDQHuFw7Va3u5533t1s9osOJa+lCHopeKbenxba0swJAOjj piv6LSGE4O8UrMucTkr8X98WljFugesqdV0q5YoNAShRdO9QH3H2qk4ytoFLTX3e4mNE alZoz6+Rc36sOCllsLQCh1e01vLkJ1U3wLn5/ygxL/oV3gjLSQh4rH0niuiVaa0P+WSJ qB9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MIJDY0wCVkf83j0A2folG6JUn+sBiPxS+WIpYPexzXA=; b=uR4AwNJDauVS9sMd6Vd9XkTrJENi3j+Amu/aEIgNiiP6Vuvpnbp9icyN9rG+aXyFgj RZH74if1COyJEXQmYAWvX/3UN6LuHU2qmL3fYA7JSnpGysakPOt75oHiM8yGEjnjR8Fr p1si31o/jEps9rH93XYOu+g9qGnpkHgsysJmRGq7ZetLuDdLpX0ltyvTzPD+bJ1Pi6xT pQ/qq6ybVpLDZI/0vjf17XKqKs70Ls5ShKuN75GbQ3SfUOrjqvCeHthcrnEQI0BBT7c6 EpZeVZb4ZQiDI//IJHdu+4ejarukrru6rcu/+leK/ey56GYAoER7y4ZKqnopT932PYaR 897g== X-Gm-Message-State: AOAM5335SdUC7Lhyc8eImoD8lG1SFrQlHkf+ER2QzrVQIEMjcOAy9+Vd aGE5IZOgcaRFKOwtW7c3AAn4bw== X-Google-Smtp-Source: ABdhPJwyKPX/7cdfW3jjDhePiHIxMw4yOMpTtCwt1bs4QdfHyqxZkGm2DmTzSfoOmdU2m+1CZcsyMg== X-Received: by 2002:adf:9404:: with SMTP id 4mr30595963wrq.367.1593007504234; Wed, 24 Jun 2020 07:05:04 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 12/25] tests/vm: switch from optsparse to argparse Date: Wed, 24 Jun 2020 15:04:33 +0100 Message-Id: <20200624140446.15380-13-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::443; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) optparse has been deprecated since version 3.2 and argparse is the blessed replacement. Take the opportunity to enhance our help output showing defaults when called. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Robert Foley --- tests/vm/basevm.py | 84 +++++++++++++++++++++++----------------------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 04d083409a5..93859362606 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -23,7 +23,7 @@ from qemu.accel import kvm_available from qemu.machine import QEMUMachine import subprocess import hashlib -import optparse +import argparse import atexit import tempfile import shutil @@ -556,47 +556,47 @@ def parse_args(vmcls): else: return 1 =20 - parser =3D optparse.OptionParser( - description=3D"VM test utility. Exit codes: " - "0 =3D success, " - "1 =3D command line error, " - "2 =3D environment initialization failed, " - "3 =3D test command failed") - parser.add_option("--debug", "-D", action=3D"store_true", - help=3D"enable debug output") - parser.add_option("--image", "-i", default=3D"%s.img" % vmcls.name, - help=3D"image file name") - parser.add_option("--force", "-f", action=3D"store_true", - help=3D"force build image even if image exists") - parser.add_option("--jobs", type=3Dint, default=3Dget_default_jobs(), - help=3D"number of virtual CPUs") - parser.add_option("--verbose", "-V", action=3D"store_true", - help=3D"Pass V=3D1 to builds within the guest") - parser.add_option("--build-image", "-b", action=3D"store_true", - help=3D"build image") - parser.add_option("--build-qemu", - help=3D"build QEMU from source in guest") - parser.add_option("--build-target", - help=3D"QEMU build target", default=3D"check") - parser.add_option("--build-path", default=3DNone, - help=3D"Path of build directory, "\ - "for using build tree QEMU binary. ") - parser.add_option("--interactive", "-I", action=3D"store_true", - help=3D"Interactively run command") - parser.add_option("--snapshot", "-s", action=3D"store_true", - help=3D"run tests with a snapshot") - parser.add_option("--genisoimage", default=3D"genisoimage", - help=3D"iso imaging tool") - parser.add_option("--config", "-c", default=3DNone, - help=3D"Provide config yaml for configuration. "\ - "See config_example.yaml for example.") - parser.add_option("--efi-aarch64", - default=3D"/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", - help=3D"Path to efi image for aarch64 VMs.") - parser.add_option("--log-console", action=3D"store_true", - help=3D"Log console to file.") - parser.disable_interspersed_args() - return parser.parse_args() + parser =3D argparse.ArgumentParser( + formatter_class=3Dargparse.ArgumentDefaultsHelpFormatter, + description=3D"Utility for provisioning VMs and running builds", + epilog=3D"""Remaining arguments are passed to the command. + Exit codes: 0 =3D success, 1 =3D command line error, + 2 =3D environment initialization failed, + 3 =3D test command failed""") + parser.add_argument("--debug", "-D", action=3D"store_true", + help=3D"enable debug output") + parser.add_argument("--image", "-i", default=3D"%s.img" % vmcls.name, + help=3D"image file name") + parser.add_argument("--force", "-f", action=3D"store_true", + help=3D"force build image even if image exists") + parser.add_argument("--jobs", type=3Dint, default=3Dget_default_jobs(), + help=3D"number of virtual CPUs") + parser.add_argument("--verbose", "-V", action=3D"store_true", + help=3D"Pass V=3D1 to builds within the guest") + parser.add_argument("--build-image", "-b", action=3D"store_true", + help=3D"build image") + parser.add_argument("--build-qemu", + help=3D"build QEMU from source in guest") + parser.add_argument("--build-target", + help=3D"QEMU build target", default=3D"check") + parser.add_argument("--build-path", default=3DNone, + help=3D"Path of build directory, "\ + "for using build tree QEMU binary. ") + parser.add_argument("--interactive", "-I", action=3D"store_true", + help=3D"Interactively run command") + parser.add_argument("--snapshot", "-s", action=3D"store_true", + help=3D"run tests with a snapshot") + parser.add_argument("--genisoimage", default=3D"genisoimage", + help=3D"iso imaging tool") + parser.add_argument("--config", "-c", default=3DNone, + help=3D"Provide config yaml for configuration. "\ + "See config_example.yaml for example.") + parser.add_argument("--efi-aarch64", + default=3D"/usr/share/qemu-efi-aarch64/QEMU_EFI.fd= ", + help=3D"Path to efi image for aarch64 VMs.") + parser.add_argument("--log-console", action=3D"store_true", + help=3D"Log console to file.") + return parser.parse_known_args() =20 def main(vmcls, config=3DNone): try: --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008465; cv=none; d=zohomail.com; s=zohoarc; b=mpueRQXgGJzwquQfwwTfvnWEpl1CJcp/Ncdm0zMpjH98GgRCkFcbAgmogZj7aFbabR/ezo1jOvbZgLcLbwE5m8fOoOijHhYhMLuiP3TdSBUnQBK4BvrtnE9ohKqhg62nP0uMIYNpW9d0sY3/sM2Nz8LJDs8F+b8UNOi4RXjLnxg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008465; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xcC1dEPRPSwS0guxyTrS7h54huqSdjkxAZKSe+0TYJU=; b=X/0oO/LIlN3ogrhcRixG4PaIWipLMb7YzXExJb+RqGxW28EwokUG+yLtqvJQ1gqppJ7OygTz013s8cG8LDAYWE4vQ/WLrJmvXgiQtlJu4EYX4f21zHT5lcPuIA/GXIEyeS/X4kKoVTK0H3CT+460cY0yJuK69bYriBd2mZuxedE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008465823138.88278016369134; Wed, 24 Jun 2020 07:21:05 -0700 (PDT) Received: from localhost ([::1]:38448 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Gu-0002Ir-H6 for importer@patchew.org; Wed, 24 Jun 2020 10:21:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59406) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61d-0005Pq-FO for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:17 -0400 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]:35785) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61X-0003Xl-Dv for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:17 -0400 Received: by mail-wr1-x442.google.com with SMTP id g18so2434093wrm.2 for ; Wed, 24 Jun 2020 07:05:10 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id o29sm19603059wra.5.2020.06.24.07.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 82D111FF9C; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xcC1dEPRPSwS0guxyTrS7h54huqSdjkxAZKSe+0TYJU=; b=H6iBPqZtj/NDjb+zP/vFXoXSJkBu4Ypbg9W0WZqrHWAZjn7mE8c56oQmkQTwU6zRw8 uK21bRZtmUv4ncvsHCBz7n2kc3ZzrhUQDgT/E58G6kEjQ/BBZqdrlewn8Ay+HAw6Ta3S 4HMyOH4PTJ7jfGm2MaJIweyAFbiNWOHb+IQEDmFpoGvvmlq4BgFSk64wOQZi2hfxdyns A7p9+8CklWqF63sne/90FO+KL4GDAZKjCrjaebYkEUuKnswOB9pdW8E3SZNdJBUlfosS UZruaHU7FrSj5k1ygj9Dc1VBEZSde8l7D7Z8rppaUOBmy0by9YEhtyRLO/5CjL2FblS2 c+NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xcC1dEPRPSwS0guxyTrS7h54huqSdjkxAZKSe+0TYJU=; b=IRKjF6ginI794f8/DVQt/Y/GSF47VTt9E1+h7Gu0I2vXwRzSB9Lcp23kvmlPnV6+Mu HejsA3bXyKukdilAXZqmV6EuF4Vy9NeW0mjq/6UanLUkOmiLeTsJ5SGisP6GTOmKTzb+ f0n2evAMid3UUZ7bV1NWTrGksLRWsTDUqGdGx8iK0wIjb6cK9gaIFDJd1ky9Ax/34/86 3v0o9Tu7gm7mLD8wetccMOTmR6tcsHYs2YqaaE/fJHA6IPgreY32T94HPSEs2z4zdYyB JS7TYwfCDs+RrCK3HsbCm9EUBKPqFt62H98uYcAhBM5xBvyRfrUPsGRzfvglMsYhL5Ct LVIw== X-Gm-Message-State: AOAM532m+erVib93DP6U7WCeNIZhGWYiDvdpYtlUuS5WeumDsFGLc0My R6cpUiRcsvuC16y0/WmzMPLmkA== X-Google-Smtp-Source: ABdhPJxgZVe55tlw9OjASmiSFYQohPDerTNt4hdeyUYML/UX1zevlHzBntIvcfCgHgYpAbGiprSYpA== X-Received: by 2002:adf:f608:: with SMTP id t8mr16419707wrp.308.1593007509905; Wed, 24 Jun 2020 07:05:09 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 13/25] tests/vm: allow us to take advantage of MTTCG Date: Wed, 24 Jun 2020 15:04:34 +0100 Message-Id: <20200624140446.15380-14-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::442; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x442.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) We currently limit TCG guests to -smp 1 but now we have added some aarch64 guests we can do better when running on x86_64 hardware. Raise the limit for TCG guests when it is safe to do so. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Robert Foley --- tests/vm/basevm.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 93859362606..dd96a6d4af6 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -553,6 +553,12 @@ def parse_args(vmcls): def get_default_jobs(): if kvm_available(vmcls.arch): return multiprocessing.cpu_count() // 2 + elif os.uname().machine =3D=3D "x86_64" and \ + vmcls.arch in ["aarch64", "x86_64", "i386"]: + # MTTCG is available on these arches and we can allow more cor= es. + # But only up to a reasonable limit. User can always override + # these limits with --jobs. + return min(multiprocessing.cpu_count() // 2, 8) else: return 1 =20 --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593007744; cv=none; d=zohomail.com; s=zohoarc; b=WUMNPZRytW3EcMERr6pOrUJSKh+FdBStK66sJglpXGLZI7RoCJKeuhcTaqFQhl4hvbnU58qM88gnZJ8e77T1+Jx6majWI0SYVtCr/c0qYd8P7hGKCU9VFSpSJ1XJvb1qZtmoodQOfcV98MGuHZvyLuJCIkWFkp2YQC3wDoGEdT8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593007744; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UsvCJ1FtgsYHjIIgU0bFzUSnvljWkIkb9Subsb2kO6E=; b=YhD4ABDDe+tQ/baIVKB0RTCouyLnBayKsYm9YlhWP0J9yibndJPqDhXg01WbsganGmx1oor4AeD+3gdlNtJJgxGjkpN0lhoF/rfdOZ/tsn8jRpJYortmanuV+SpykDthhK0EKr3DTjoxY11X1cc9E70w32TemwskV0aZdwEWth8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593007744124669.7357240524037; Wed, 24 Jun 2020 07:09:04 -0700 (PDT) Received: from localhost ([::1]:50916 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo65H-0005hn-5t for importer@patchew.org; Wed, 24 Jun 2020 10:09:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59340) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61Y-0005Gw-G6 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:12 -0400 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]:36748) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61W-0003Xa-K4 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:12 -0400 Received: by mail-wm1-x342.google.com with SMTP id 17so2665057wmo.1 for ; Wed, 24 Jun 2020 07:05:10 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id l17sm8465200wmh.14.2020.06.24.07.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 991891FF9D; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UsvCJ1FtgsYHjIIgU0bFzUSnvljWkIkb9Subsb2kO6E=; b=mhtkAArBeJ4g/pLN2chI6H/E73MmkTKkwmTatlo/mA/guux3+weGLovZ++icakMoyU j+2Khfas98ct2bAFbqzfLPTkGalXvcq9qPkbSiB/i9oeD9bH5IMUkYMQbnmt4IMYoGgF 4vZ+NpQck/C54duEym+T2Xja+7gLN3Y8ReT7GJo0Ckuj3HxY4Z24B3mSVrY6cN0OEGSF aFAAG8UBU9NyshKmDHc3UZ6rE2KQzv1shF6Sxlfq5j06q+7GzM+WD/PMxftv3FFcr2NF LglPvuO4Vc9zd4ppau6bLo0IOvcT19oDv2xJmxWhalXyGxpgLs63uXUSiQGXLoHZpR26 O6aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UsvCJ1FtgsYHjIIgU0bFzUSnvljWkIkb9Subsb2kO6E=; b=OOd8Ec7NwXT+CSZO96YS9gVpHu3cwVfpBPfo+PNFujJGzeCh+FM5IAjS+USYj8R9HO 6NAPJY3vBFvq3+/3Zz39VD4zNSIUA9ScQtGgMDJRELuM9fTAnXdtRkaIPxWZt6O/R/8t r7MEB+9rkPd+ZFpRoCm+sFXqeSTzt9QGuo1UX5VUDtsp0eLUxSKr0VuAnFx1Py8QAouP 1hoJzlLQqr4HBPeSQm7WbDZBLtNgmuqt5FXF/DfYuwBJAESQ0GH5wyXGNDz6dfMZl6NW zfEkP5FDDfvQpFRkjynxRwpAcg6ZPxDcyQhDryyr9VaJDvmvqDiH5jq0zA09HY0ZORD4 xPGQ== X-Gm-Message-State: AOAM532L3SVSnOOQblvY6f/08JRJSSB2F9rVYpUpwvIuqwUHsbmnVATN RcbwPWxcHELgw+vjI5YOLoaM9g== X-Google-Smtp-Source: ABdhPJzN4Km5lIe5ISVFN6u6iefkkTo7nmzWOEl8kbQWiqaLaTmO05JjXE1D2lPpIKs80x7jHDq/RQ== X-Received: by 2002:a1c:1946:: with SMTP id 67mr31731088wmz.59.1593007509054; Wed, 24 Jun 2020 07:05:09 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 14/25] tests/docker: check for an parameters not empty string Date: Wed, 24 Jun 2020 15:04:35 +0100 Message-Id: <20200624140446.15380-15-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::342; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x342.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Peter Maydell , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Reported-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Alex Benn=C3=A9e Suggested-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/docker/common.rc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/common.rc b/tests/docker/common.rc index 02cd67a8c5e..b27ce17e333 100755 --- a/tests/docker/common.rc +++ b/tests/docker/common.rc @@ -47,7 +47,7 @@ build_qemu() check_qemu() { # default to make check unless the caller specifies - if test -z "$@"; then + if [ $# =3D=3D 0 ]; then INVOCATION=3D"check" else INVOCATION=3D"$@" --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008250; cv=none; d=zohomail.com; s=zohoarc; b=LYUT3tXXclFSaIMPeBHTR+9pzVu7njH+wi22NKhCzUZ0tKHqZvKZTq6rwKwlS33mZQJzSJEMNuP7lCxy0qre5jdXVqTzqMfky0wG10D/FCvQ2XA56wx6eoqq1JL2ORdrrRox6wYHxnrPyr1xtP1PLc+Uq9RXlvDJouwLFveyD6c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008250; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OeJNj8ZC3zV8CkJ1XiE1kv6/qIn2cm2cQAQo+Livlkg=; b=bdmqu74XuV4PvPCVBs4lV98VjbA9UUVJRekh4tf15i5Iit2G+g5SlONQUI3sFEbkTngP816AXrUN73ZASZDoR9jAwaiP+6BfMAX9cah57v561R9rTZtWXaiJ2uOQsaXoZ+kqTWeqEm1QyKa9LA0NFzDhzmLW6tjdzZoN9scYcMY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008250199185.05127388062579; Wed, 24 Jun 2020 07:17:30 -0700 (PDT) Received: from localhost ([::1]:53230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6DQ-0004C2-LE for importer@patchew.org; Wed, 24 Jun 2020 10:17:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61a-0005LK-VR for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:15 -0400 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:35006) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61U-0003XG-M8 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:14 -0400 Received: by mail-wm1-x32a.google.com with SMTP id l2so1174056wmf.0 for ; Wed, 24 Jun 2020 07:05:08 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id h14sm11339731wrt.36.2020.06.24.07.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id D9A231FF9E; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OeJNj8ZC3zV8CkJ1XiE1kv6/qIn2cm2cQAQo+Livlkg=; b=mdjXnnwr4AEoSGvM85xMNKZwqFsizhFDD96walh/C77iUjymYsig5jydQ5K0mZKJO1 fGPZfCYB1hQi7QyeD/BBBmZJgua+z7pXPCkYSzSxWFwDza5Tw7KaXSxx2/8Y4sKbnhgh G4oee14IJVDe2dVFCaFyxc2lywyVxB+FmmIl3ToveN/+x4nkMLFVRmH+SY9Hd63mauUR 4SacVMtg3S+EJb3+lT94cjRTz7pSAj2y4z7Wx1WxjlWWiETju6OcYSMEI46OdiUfmSyL 9upTi6QtHzW+IV8XB2rV2BJpZ79xkQrWIwwYeWmUt4DtqKX0XMGfrNr60730nUyklQm1 OiWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OeJNj8ZC3zV8CkJ1XiE1kv6/qIn2cm2cQAQo+Livlkg=; b=Eq3vMt7LtxJxlJuj5LwQjjySbUoDy6eaR7xDLv7wfMDf2enz+LhYYnZ5DLtRZ1qPGt cMGvekTvbt+sxu/FP8jc7PvKARakwDaQ/gL0/NYKBa2o1Y6dMCfVffv350mKnCQC7GBB JKiauMam/enuNBzWXqYc6VWQYw2R4cY4z/LOHNxG+rXVQH83nr1ayfMJSFNW9vjK6oUn qoByWBLFa9MmuUBpLzIfImsn/AqaCB+gZ5E7bFiqsUn5giv1rpTvTezaRPXvr5DCxczs zJDdGwvqvqDukbZQX13yDPUlH1gghuN56fB4AI6cjOfzHTptOsfRrlZ0h4qQs2TTABGv wsLg== X-Gm-Message-State: AOAM533QtWy6h05e4gPjFcaInaHy8UmQhup67psj5pd1/ZBwCy8EIBJU Rtcjwnk0PR9VXJxYytUISmzrgA== X-Google-Smtp-Source: ABdhPJxoQcqhAa6507gSJ+tIMYVkDxikC594U+eec6qHiwD1cIxbOX5vabqaZE3lprt6WJ7H0DH0AA== X-Received: by 2002:a05:600c:218f:: with SMTP id e15mr23451277wme.187.1593007507061; Wed, 24 Jun 2020 07:05:07 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 15/25] tests/docker: change tag naming scheme of our images Date: Wed, 24 Jun 2020 15:04:36 +0100 Message-Id: <20200624140446.15380-16-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) We've been misusing the tag naming scheme for some time by overloading the post : section with the image type. Really it should be saved for the revision of that particular build. Move the details to the other side so we have: qemu/image-name with the implied :latest version added by the tooling. Suggested-by: Daniel P. Berrang=C3=A9 Signed-off-by: Alex Benn=C3=A9e --- tests/docker/Makefile.include | 2 +- tests/docker/docker.py | 2 +- tests/docker/dockerfiles/debian-alpha-cross.docker | 2 +- tests/docker/dockerfiles/debian-amd64-cross.docker | 2 +- tests/docker/dockerfiles/debian-amd64.docker | 2 +- tests/docker/dockerfiles/debian-arm64-cross.docker | 2 +- tests/docker/dockerfiles/debian-arm64-test-cross.docker | 2 +- tests/docker/dockerfiles/debian-armel-cross.docker | 2 +- tests/docker/dockerfiles/debian-armhf-cross.docker | 2 +- tests/docker/dockerfiles/debian-hppa-cross.docker | 2 +- tests/docker/dockerfiles/debian-m68k-cross.docker | 2 +- tests/docker/dockerfiles/debian-mips-cross.docker | 2 +- tests/docker/dockerfiles/debian-mips64-cross.docker | 2 +- tests/docker/dockerfiles/debian-mips64el-cross.docker | 2 +- tests/docker/dockerfiles/debian-mipsel-cross.docker | 2 +- tests/docker/dockerfiles/debian-powerpc-cross.docker | 2 +- tests/docker/dockerfiles/debian-ppc64-cross.docker | 2 +- tests/docker/dockerfiles/debian-ppc64el-cross.docker | 2 +- tests/docker/dockerfiles/debian-riscv64-cross.docker | 2 +- tests/docker/dockerfiles/debian-s390x-cross.docker | 2 +- tests/docker/dockerfiles/debian-sh4-cross.docker | 2 +- tests/docker/dockerfiles/debian-sparc64-cross.docker | 2 +- tests/docker/dockerfiles/debian-tricore-cross.docker | 2 +- tests/docker/dockerfiles/debian-win32-cross.docker | 2 +- tests/docker/dockerfiles/debian-win64-cross.docker | 2 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 26 files changed, 26 insertions(+), 26 deletions(-) diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 3e3617816ec..7bac1a67e3e 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -55,7 +55,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker else docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command,\ - $(DOCKER_SCRIPT) build -t qemu:$* -f $< \ + $(DOCKER_SCRIPT) build -t qemu/$* -f $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ diff --git a/tests/docker/docker.py b/tests/docker/docker.py index e630aae1089..cc6f76caa60 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -204,7 +204,7 @@ def _dockerfile_preprocess(df): for l in df.splitlines(): if len(l.strip()) =3D=3D 0 or l.startswith("#"): continue - from_pref =3D "FROM qemu:" + from_pref =3D "FROM qemu/" if l.startswith(from_pref): # TODO: Alternatively we could replace this line with "FROM $I= D" # where $ID is the image's hex id obtained with diff --git a/tests/docker/dockerfiles/debian-alpha-cross.docker b/tests/doc= ker/dockerfiles/debian-alpha-cross.docker index 74bcabfdb12..10fe30df0dd 100644 --- a/tests/docker/dockerfiles/debian-alpha-cross.docker +++ b/tests/docker/dockerfiles/debian-alpha-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-amd64-cross.docker b/tests/doc= ker/dockerfiles/debian-amd64-cross.docker index 5d890419251..870109ef6af 100644 --- a/tests/docker/dockerfiles/debian-amd64-cross.docker +++ b/tests/docker/dockerfiles/debian-amd64-cross.docker @@ -4,7 +4,7 @@ # This docker target is used on non-x86_64 machines which need the # x86_64 cross compilers installed. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Alex Benn=C3=A9e =20 # Add the foreign architecture we want and install dependencies diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/do= ckerfiles/debian-amd64.docker index 957f0bc2e79..8fdfd6a6b04 100644 --- a/tests/docker/dockerfiles/debian-amd64.docker +++ b/tests/docker/dockerfiles/debian-amd64.docker @@ -4,7 +4,7 @@ # This docker target builds on the debian Stretch base image. Further # libraries which are not widely available are installed by hand. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 RUN apt update && \ diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/doc= ker/dockerfiles/debian-arm64-cross.docker index 09ca0a1ba79..b1a4613da5f 100644 --- a/tests/docker/dockerfiles/debian-arm64-cross.docker +++ b/tests/docker/dockerfiles/debian-arm64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10:latest =20 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture arm64 diff --git a/tests/docker/dockerfiles/debian-arm64-test-cross.docker b/test= s/docker/dockerfiles/debian-arm64-test-cross.docker index a44e76d9421..53a9012beb8 100644 --- a/tests/docker/dockerfiles/debian-arm64-test-cross.docker +++ b/tests/docker/dockerfiles/debian-arm64-test-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Bullseye base image. # -FROM qemu:debian11 +FROM qemu/debian11 =20 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture arm64 diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/doc= ker/dockerfiles/debian-armel-cross.docker index e3794a61c95..b7b1a3585f8 100644 --- a/tests/docker/dockerfiles/debian-armel-cross.docker +++ b/tests/docker/dockerfiles/debian-armel-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 # Add the foreign architecture we want and install dependencies diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/doc= ker/dockerfiles/debian-armhf-cross.docker index e163b8b9569..25d76188337 100644 --- a/tests/docker/dockerfiles/debian-armhf-cross.docker +++ b/tests/docker/dockerfiles/debian-armhf-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture armhf diff --git a/tests/docker/dockerfiles/debian-hppa-cross.docker b/tests/dock= er/dockerfiles/debian-hppa-cross.docker index 5c68b2d3304..3d6c65a3efc 100644 --- a/tests/docker/dockerfiles/debian-hppa-cross.docker +++ b/tests/docker/dockerfiles/debian-hppa-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-m68k-cross.docker b/tests/dock= er/dockerfiles/debian-m68k-cross.docker index 25edc80e9a3..fcb10e35347 100644 --- a/tests/docker/dockerfiles/debian-m68k-cross.docker +++ b/tests/docker/dockerfiles/debian-m68k-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-mips-cross.docker b/tests/dock= er/dockerfiles/debian-mips-cross.docker index 08a8e1c29cc..26c154014db 100644 --- a/tests/docker/dockerfiles/debian-mips-cross.docker +++ b/tests/docker/dockerfiles/debian-mips-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 diff --git a/tests/docker/dockerfiles/debian-mips64-cross.docker b/tests/do= cker/dockerfiles/debian-mips64-cross.docker index 1a79505d696..09c2ba584e5 100644 --- a/tests/docker/dockerfiles/debian-mips64-cross.docker +++ b/tests/docker/dockerfiles/debian-mips64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/= docker/dockerfiles/debian-mips64el-cross.docker index 453b53ef72c..c990b683b7a 100644 --- a/tests/docker/dockerfiles/debian-mips64el-cross.docker +++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker @@ -4,7 +4,7 @@ # This docker target builds on the debian Stretch base image. # =20 -FROM qemu:debian10 +FROM qemu/debian10 =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/do= cker/dockerfiles/debian-mipsel-cross.docker index 3b6e975c686..0e5dd42d3c4 100644 --- a/tests/docker/dockerfiles/debian-mipsel-cross.docker +++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 diff --git a/tests/docker/dockerfiles/debian-powerpc-cross.docker b/tests/d= ocker/dockerfiles/debian-powerpc-cross.docker index 89dd4fbf870..07e1789650b 100644 --- a/tests/docker/dockerfiles/debian-powerpc-cross.docker +++ b/tests/docker/dockerfiles/debian-powerpc-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-ppc64-cross.docker b/tests/doc= ker/dockerfiles/debian-ppc64-cross.docker index 4bf88ab02dd..8efe68874ed 100644 --- a/tests/docker/dockerfiles/debian-ppc64-cross.docker +++ b/tests/docker/dockerfiles/debian-ppc64-cross.docker @@ -2,7 +2,7 @@ # Docker ppc64 cross-compiler target # # This docker target builds on the debian Buster base image. -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/d= ocker/dockerfiles/debian-ppc64el-cross.docker index cd386f01d93..1146a06be62 100644 --- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker +++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture ppc64el && \ diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/d= ocker/dockerfiles/debian-riscv64-cross.docker index 5e2d6ddb600..2bbff19772a 100644 --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-s390x-cross.docker b/tests/doc= ker/dockerfiles/debian-s390x-cross.docker index 43fe59836f2..9f2ab51eb09 100644 --- a/tests/docker/dockerfiles/debian-s390x-cross.docker +++ b/tests/docker/dockerfiles/debian-s390x-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 # Add the s390x architecture RUN dpkg --add-architecture s390x diff --git a/tests/docker/dockerfiles/debian-sh4-cross.docker b/tests/docke= r/dockerfiles/debian-sh4-cross.docker index 9d7663764e4..fd3af895759 100644 --- a/tests/docker/dockerfiles/debian-sh4-cross.docker +++ b/tests/docker/dockerfiles/debian-sh4-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-sparc64-cross.docker b/tests/d= ocker/dockerfiles/debian-sparc64-cross.docker index 31fd34f120b..f4bb9b561cf 100644 --- a/tests/docker/dockerfiles/debian-sparc64-cross.docker +++ b/tests/docker/dockerfiles/debian-sparc64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 =20 RUN apt update && \ DEBIAN_FRONTEND=3Dnoninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-tricore-cross.docker b/tests/d= ocker/dockerfiles/debian-tricore-cross.docker index 4a0f7706a39..769d95c77bb 100644 --- a/tests/docker/dockerfiles/debian-tricore-cross.docker +++ b/tests/docker/dockerfiles/debian-tricore-cross.docker @@ -7,7 +7,7 @@ # # SPDX-License-Identifier: GPL-2.0-or-later # -FROM qemu:debian9 +FROM qemu/debian9 =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/doc= ker/dockerfiles/debian-win32-cross.docker index d16d6431bc8..b045e821b91 100644 --- a/tests/docker/dockerfiles/debian-win32-cross.docker +++ b/tests/docker/dockerfiles/debian-win32-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch MXE base image. # -FROM qemu:debian9-mxe +FROM qemu/debian9-mxe =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/doc= ker/dockerfiles/debian-win64-cross.docker index b0bc960445b..2fc9cfcbc66 100644 --- a/tests/docker/dockerfiles/debian-win64-cross.docker +++ b/tests/docker/dockerfiles/debian-win64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch MXE base image. # -FROM qemu:debian9-mxe +FROM qemu/debian9-mxe =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 diff --git a/tests/docker/dockerfiles/debian9-mxe.docker b/tests/docker/doc= kerfiles/debian9-mxe.docker index 62ff1cecf2d..ae2c222a6f0 100644 --- a/tests/docker/dockerfiles/debian9-mxe.docker +++ b/tests/docker/dockerfiles/debian9-mxe.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian9 +FROM qemu/debian9 =20 MAINTAINER Philippe Mathieu-Daud=C3=A9 =20 --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008126; cv=none; d=zohomail.com; s=zohoarc; b=fiuA99+MjEDa0cVlHf8MJtYN31aSCZwdd5HyRr3zDyejXqU9hE77F0uP75itDbirjuvIgGUVxPhO4/hh+EFgpCtEHYv5MapYmtPaeqBRSS2n6Ni1rm8hwiLXKV7wz/oh3XO6HMEwtGLEd1crI4nR7vEkhv1stjfEuTTp0Q+ODcU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008126; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XJD1GL90UWNycRg4BT4TsZXTpQO4+SzPpVo+RlIQlho=; b=Rrg3bxZzKXUj/47AVcFhWpnuzgKrTLwBFEeYedCa2u4USbqHFtXVKxxk4J0rwPBCJ8Txjuxmyc3dNF/Z8r3KmDIHpDZRFaWi4bC8LT3F5HkU2IgSp90qLvjfkHUD/5FFxYUxla2oy4Dxt5wuaz16faygsfw7dDsp2UG7JCKcePY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008126101585.1745074964649; Wed, 24 Jun 2020 07:15:26 -0700 (PDT) Received: from localhost ([::1]:44752 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6BR-0000Qk-5K for importer@patchew.org; Wed, 24 Jun 2020 10:15:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59322) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61X-0005EL-0J for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:11 -0400 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:40721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61V-0003XK-8A for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:10 -0400 Received: by mail-wr1-x443.google.com with SMTP id h5so2416053wrc.7 for ; Wed, 24 Jun 2020 07:05:08 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id i15sm27161336wre.93.2020.06.24.07.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id EE1891FF9F; Wed, 24 Jun 2020 15:04:47 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XJD1GL90UWNycRg4BT4TsZXTpQO4+SzPpVo+RlIQlho=; b=qoVgHKhlnZMVtwgby0vh1A08Lu4m3eq7kDoxQxMuTRN05QuT8ASII+v2D83VXiJ2KH uItqonBdKB++2jl2DsS8U2/G1eO8HV9YKbimwSBr0H4uCuDuCg021y2GKHJ+opXRS4mK XXWox5tELn4s4QMvE+VpH2+J0vmuDJH/06s4+gnbvwkafQ5wCZp+9XPNLqylzc5sQWLN d9X7sV3Ax/DamLuedPt70f9NxiMRbXDl0kncC4rh+jnn0UPBBQUnWPWtkvwirL4lhaMS k0NDc9NzACSh1HVo+RwtltZAWF5DGKf/65xabbQnSOiNYEAOB7Xv8VIRZG53JK2G3kuB 1avQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XJD1GL90UWNycRg4BT4TsZXTpQO4+SzPpVo+RlIQlho=; b=af8FCoZ/bRaIPtOIriogm/7AGIu07TzpT9U768kW7E6FrWw98MigQcfswLJmx11j4h yf/aRBl93Z9Mv52qyfdphbNhMi48G5eIfGLtQLsQ+S7BWlmc/2OViGvgXnEYe+Q8dwSE dEtf8tT361FnM/cGpZezdLsqRUzt/DnIbQ6JMeip8Nfn0/yKVJazCjIi/Ny7c0NU5ssD +zhnFerLy09OxFfgd+GwXbxTtsVuVA9t9ieuL3at65JNTBPRdtA32+himTbN6uquSokn XsLWhuX68iFSTn8L/9jx1tNZCpMSaRE7vddcHhdOnOVhMJUHiVWld1/gcnsdN2sm9I90 +tAw== X-Gm-Message-State: AOAM532ZdhFAGm9fvoZH73aPOsB6tg25ffGJ7Olw8NEdRsY5u99N+2Nt zilzGii2wMSllkq+G+eug9/WlQ== X-Google-Smtp-Source: ABdhPJzJMOBcqSmaD0RE5oXCciOuPB2Ho63lxIX27IS9k6JifKDS83LIFQe9hSzaAdG8mS/t36Cq/w== X-Received: by 2002:adf:e38d:: with SMTP id e13mr10562160wrm.304.1593007507864; Wed, 24 Jun 2020 07:05:07 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 16/25] .gitignore: un-ignore .gitlab-ci.d Date: Wed, 24 Jun 2020 15:04:37 +0100 Message-Id: <20200624140446.15380-17-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::443; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) The sooner we deprecate in-tree builds the sooner this mess of regexes can be thrown away. Signed-off-by: Alex Benn=C3=A9e --- v2 - just use explicit !/.gitlab-ci.d --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 90acb4347d4..2992d15931a 100644 --- a/.gitignore +++ b/.gitignore @@ -93,6 +93,7 @@ *.tp *.vr *.d +!/.gitlab-ci.d !/scripts/qemu-guest-agent/fsfreeze-hook.d *.o .sdk --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008171; cv=none; d=zohomail.com; s=zohoarc; b=Lvhe0x7FZDtd+uvzam1z+87AgnwlXKPgVPq4TBwjnwd1QTASmYdgS0B+d/jvvapGlJgsVh8HNM/yuQk/A8Cqm58XyiiOL1kVXDomHoL5pE2Eos0zAkDUgG8pLOs3rC/1xXjmQVmQa8S9TxpXkIvLOtfR/unK5JaupQU8sFmVSV4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008171; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c7AoJ4KTaY68imxRMzTzv8EVx07Om13RK/P3Xy39tRA=; b=YeD0Kx7XqKy9zkczRgIpYWv6ewMetXbsMUoU8LKjmr5SHzUuHlXeM5XdsHdrfpE5iXhWq+ZMt/L8tbPsGWamlU7q3dtuiFnLfIkdV4GprCjSdMwortZZRLPQNAlXYj2iRYy0uTQK1yiYzVhc1yUuJcNM3BW5prmTX+qjeOs5MGM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159300817158640.87796517188781; Wed, 24 Jun 2020 07:16:11 -0700 (PDT) Received: from localhost ([::1]:47970 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6CA-00023t-Nb for importer@patchew.org; Wed, 24 Jun 2020 10:16:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68v-0004AC-Cr for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:49 -0400 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:35448) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68t-0005vz-If for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:49 -0400 Received: by mail-wr1-x42b.google.com with SMTP id g18so2461547wrm.2 for ; Wed, 24 Jun 2020 07:12:47 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id c25sm8081800wml.46.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 1500B1FFA5; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c7AoJ4KTaY68imxRMzTzv8EVx07Om13RK/P3Xy39tRA=; b=xRVAKtGzctVjzoQCyfyZRkdeeUi2NkP5Issuk3qYnS3FbpxRyGuXNBvBmHHYenzAvH fC+Dn+5oQ+oCb+ipGsAcPTRNOTKLwcwAXpIeipS3+hvyoRvVMaEftrdv1nb8d/t6Pnp8 v1gHA41TJeN344UqvquKepl+L/ONSiXMAz8GPLoiC2IbEbqrf8XfmmfqGNkX/FrtXwKx q7o392Wk+RGfKyTDxRxu61GXZU+PWy4AwOQl6dUcYKd8QzpVtPFTeXKwukT7YjivDyG6 V6lFpasCGfJIoxBljMVMi6hzjdN/OVLORWSKBwEk75DCAFBvp949mPtFYL3k6VERthg0 xZPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c7AoJ4KTaY68imxRMzTzv8EVx07Om13RK/P3Xy39tRA=; b=OvlLxZQxDdOX66q83dyrIclxCdn3PAcb/ECT8aJ83Qv5IYjjlNP7TZRe1kzvOO0lFZ IrY0z5ZQx/Gx+oW3IP9CFEEzgKORk982aVi38K5Vh5f5J8XvTpD8l4Gh8Hu+226y5h+i 4E83DR/vqNMhX/2n+tonybz1704t+vexVKVNmmvAii7f9ITc2elziBzoN0CLzE1ltkhi P8Yw5FTujGa6RBpchqj7kHtf63JEXmiiuvmP7FQTNw8Npo12j+V6ngM3vhEB8GzCyZi5 ZBiqbHrcuRGkf+mk6TSFjzGJ06VGp8EaNq7mqXmA3BPLrk4zSdBKNDDTaYZhXQ0PoZiA GCwg== X-Gm-Message-State: AOAM532H1nI3LL/rp/JPQ4qNkV3fAuvJX+bfWjAqWCAU+W55XvyNxZti 7xw1BeRoyp1YdKCtNOUTUfLbWA== X-Google-Smtp-Source: ABdhPJym/4S2zuCcNVyqfHLx69Rc+4tOB3VDX5FiMxt1Qtzq293tzR59rHFvn0DL8fUJcF0jUqHoNw== X-Received: by 2002:adf:f70e:: with SMTP id r14mr33237282wrp.150.1593007966044; Wed, 24 Jun 2020 07:12:46 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 17/25] gitlab: introduce explicit "container" and "build" stages Date: Wed, 24 Jun 2020 15:04:38 +0100 Message-Id: <20200624140446.15380-18-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, Laszlo Ersek , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Alex=20Benn=C3=A9e?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Daniel P. Berrang=C3=A9 If no stage is listed, jobs get put in an implicit "test" stage. Some jobs which create container images to be used by later stages are currently listed as in a "build" stages. Signed-off-by: Daniel P. Berrang=C3=A9 Acked-by: Laszlo Ersek Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200622153318.751107-2-berrange@redhat.com> --- .gitlab-ci.d/edk2.yml | 3 ++- .gitlab-ci.d/opensbi.yml | 3 ++- .gitlab-ci.yml | 11 +++++++++++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml index 088ba4b43a3..d4e7dfcba65 100644 --- a/.gitlab-ci.d/edk2.yml +++ b/.gitlab-ci.d/edk2.yml @@ -1,5 +1,5 @@ docker-edk2: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci-edk2.yml @@ -24,6 +24,7 @@ docker-edk2: - docker push $IMAGE_TAG =20 build-edk2: + stage: build rules: # Only run this job when ... - changes: # ... roms/edk2/ is modified (submodule updated) - roms/edk2/* diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index dd051c01245..ec1c1f4cab7 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -1,5 +1,5 @@ docker-opensbi: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci-opensbi.yml @@ -24,6 +24,7 @@ docker-opensbi: - docker push $IMAGE_TAG =20 build-opensbi: + stage: build rules: # Only run this job when ... - changes: # ... roms/opensbi/ is modified (submodule updated) - roms/opensbi/* diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 349c77aa580..9fdc752ea63 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,3 +1,7 @@ +stages: + - containers + - build + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' @@ -17,6 +21,7 @@ include: - JOBS=3D$(expr $(nproc) + 1) =20 build-system1: + stage: build image: ubuntu:19.10 <<: *before_script_apt script: @@ -31,6 +36,7 @@ build-system1: - make -j"$JOBS" check =20 build-system2: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -46,6 +52,7 @@ build-system2: - make -j"$JOBS" check =20 build-disabled: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -62,6 +69,7 @@ build-disabled: - make -j"$JOBS" check-qtest SPEED=3Dslow =20 build-tcg-disabled: + stage: build image: centos:8 <<: *before_script_dnf script: @@ -82,6 +90,7 @@ build-tcg-disabled: 260 261 262 263 264 270 272 273 277 279 =20 build-user: + stage: build <<: *before_script_apt script: - mkdir build @@ -92,6 +101,7 @@ build-user: - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user =20 build-clang: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -106,6 +116,7 @@ build-clang: - make -j"$JOBS" check =20 build-tci: + stage: build image: centos:8 <<: *before_script_dnf script: --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008320; cv=none; d=zohomail.com; s=zohoarc; b=i92QPgSjVRpMeu4vg8LkzovzrX/3EX1ztRIyrIx6v/m9N54QXc8OsB+WxXRTGIbxmNOj3Kc19bBn7wirpHPR2QdnyrWVpRDQB26Mrr2ft1UiGgvR2CKkH+Hb7s/4tHPT8d07m1qQtZDJbL4m5PQWODcXwbkZVKUkKPt/0qhh/t4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008320; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=P8IkpTRNJ3PGC7HEkDEmEgfBzYN/1uDvkJpY5WTdQRppAqkMQpMMe94DRbfaA4rSF/dmIUtV1+Q+xn8jKE4aKZXMseC+lmndvxjHC77xO7Wvs6PVPmloVn07gAA+diAPN2F+foh/ipD7B9znb3XGlVfExQBs/2ewAz8hUrMmvwQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008320412979.3936101123608; Wed, 24 Jun 2020 07:18:40 -0700 (PDT) Received: from localhost ([::1]:59154 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6EY-0006o0-O3 for importer@patchew.org; Wed, 24 Jun 2020 10:18:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo690-0004MG-7c for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:54 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:46103) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68y-0005yx-2O for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:53 -0400 Received: by mail-wr1-x433.google.com with SMTP id r12so2404322wrj.13 for ; Wed, 24 Jun 2020 07:12:51 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z1sm18759179wru.30.2020.06.24.07.12.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 2FCFD1FFA6; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=k4Tsq37pNmIh4FQGQktJmIeM+22h8iZzyUuLGzPNnPZSff+zV+vOm2yBPRSk+yj8qq djBgFexMBGJVdnkuiemFkYZjSm9oUsUNexWwdDVeb+s61XxWjQpdw0A+hcZOuF7r50vR Cp0kMz0bgJWGG3CHw1MwadXGfzLi7Rm3tRz1Y/NkeJt/ykShFkBFBceQcAerq1hifHWB jPpzWRq/lP2ffM9LZaeBaOCClymjZEOIz8Bty7LV39FhQ/JNCQtheD/CObAs2HUk0Hfs MldG9SbtxVJKq6Ggcxq+N8tDanl0SEBVzOyrOu8LgIEco8EC8QmCosjviKiNSlIWWTd2 O6yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=WZt/lLDU+ikOaxTaQ5/S+0otsBP37tyt6H6gRLoS+IIoCCS4D3a4FNtU21b+tCCaen 1VChqSal+ZwW2OqB16vYgydKy5yki0lgoj/W7BbOokyjoMOsRKExQDu+BomKYGStuxuv EdQruDy40rQXAO3c7/kI9g4esIebp1YQGRm+AtWFVmhvkmGGLKeOFtuCZa4F3MhW2dxs ZwDQkCtHoCtFXL1uKFlFEx7CZRBqQo+eQ37zdgkpcdYEfdAxhwXmO4tAf0586u2PPbok +6IUQ9ouz5RLDoP8oVuAB54eJLmzCZ5g1yT86qgULvSkoA+WJWX5QJjsvqGsp/GwKRlY boMQ== X-Gm-Message-State: AOAM532VasMVUlo71VOSMx5oAfp36km/7TFt5jPZucxG3nX9SszCXzJH Nft/+v98TxojcUj8wpXBm7SuCw== X-Google-Smtp-Source: ABdhPJzDqCRci6DBBk9o49XmCkA2mMdHcxjwn/7XcrfZ/DKMYWP+5m6wf/UlSLwPnODnXxG1Mb2LWQ== X-Received: by 2002:adf:e50a:: with SMTP id j10mr33329426wrm.71.1593007970435; Wed, 24 Jun 2020 07:12:50 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 18/25] gitlab: build all container images during CI Date: Wed, 24 Jun 2020 15:04:39 +0100 Message-Id: <20200624140446.15380-19-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::433; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Daniel P. Berrang=C3=A9 We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Benn=C3=A9e --- .gitlab-ci.d/containers.yml | 248 ++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 3 + 2 files changed, 251 insertions(+) create mode 100644 .gitlab-ci.d/containers.yml diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 00000000000..580439647ef --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,248 @@ + + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG=3D"$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG=3D"$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGI= STRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker= /dockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$= TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu + diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea63..72d688875fd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build =20 include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' =20 .update_apt_template: &before_script_apt before_script: --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008060; cv=none; d=zohomail.com; s=zohoarc; b=HlqaeU1ss5zuwnlBiE9G67e0pDwKmD8uOeuwUwFB89VYUDExxZjcLgxa7LSx3he+oL7LIAJGM0CnoIlfEzPByD2XF7Bc8ID9q2WWPGH+1Eo8k6HKA57Jl09pMfOX02EX3t1vKNV4xhUI/QJwdWoS5U3+TZlOk+sfWnqLak/TFSU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008060; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gYny8RQMLGvnYX7pxE0cYv3y41J+K9oraa4mW6ajsxI=; b=CPbkd90RzK1oMjxoJG/AjuFxGpI86giMAufHwoYGBjGnZeK7Lr/w3tpwBBpQdQbfj47aNBXDeAZLA3S6wBlFDeEKoGcCsWOUCH+MTIGs5O4bPrv9QC7R+EmN/jjNTScnfDwnKfHQyyA12nOtfT//ed+/wdENUL1lz2kJxYdsJ64= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008060631838.1342899100781; Wed, 24 Jun 2020 07:14:20 -0700 (PDT) Received: from localhost ([::1]:39330 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6AN-0005wx-Lb for importer@patchew.org; Wed, 24 Jun 2020 10:14:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68t-00047D-Ot for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:47 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:38193) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68r-0005v0-KQ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:47 -0400 Received: by mail-wm1-x333.google.com with SMTP id f18so2682240wml.3 for ; Wed, 24 Jun 2020 07:12:45 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id k11sm24741091wrd.23.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 457EE1FF7E; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gYny8RQMLGvnYX7pxE0cYv3y41J+K9oraa4mW6ajsxI=; b=Gd5Vpp7L+pxJ1UyhJa7Iayqcx7za3wgVZYEPEFRIvIP183AjkWg19BllD8Fbw3n02A YAaT+g4ocyAqkcGOWbpfmrqTzRb1ndTDrhI1yk0J0kKuvsHvSv41GuOE3U4v5lLCdVea AlK1n50e6+wSS7Fg5F61TsfvFVeF7QidmI33/ab8laR3rHhBZFkxi8stGY45Czs1Uc7c LoV2hdW5R/T1DHD2h5TX1HEkAce6ZvnoM0atl0T0MhVcsFIx52E2TjAL5tk048TzdQr3 ksZVpIwA5AK30jVtR7GXiHH1sYKIq2+7k213WtF6CQFnoDYgAv1JfTfmpYJPzDcr9Q3q U4BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gYny8RQMLGvnYX7pxE0cYv3y41J+K9oraa4mW6ajsxI=; b=D1ldMzNmibpH5Slpf2RYx9pNtkwb05YbVKG1L3MMaXa4lL4Kfb17oJeGKPBFoMKM0o VAIuWH0sgrDHDyIKlTgCR4Ra1H27xmeCZSS6gUj/fJhFNyhUoWV+pBCi7XVo4C9g4R6i n1B+31detmRxrtD+PmjPcqJTC4tjTTjHAvJyFER5MVJ7oI3/+r+QNR+kPL0ba3i6pvr2 75738uGk6V9s8lUzUR7y/VzPyLvGxeYz5Z5FDtz0+fgS+MfP4830TsKSNnNTdShrkzhq jXs82uO1igZeG84GTaIDRH8V+/muknMVt6sDrQxSIDy31FLk4j0V/Pi1pjHS59kekZ8I 2Upg== X-Gm-Message-State: AOAM531LFQjWIt8cHnKy0MW0SZodvEks1hvz8lD1ndJnNuWlVXYH5MWx L4K4cPr3I557Rp8KoUdb5nXj+w== X-Google-Smtp-Source: ABdhPJxi3TpJlhUdAD+Hn8p6ahxt8mhOWRseW9KYp6QyFRn31Awy7l16EXg0gYbKnMy4k1IWzgzBkg== X-Received: by 2002:a7b:c1cc:: with SMTP id a12mr21586291wmj.112.1593007963596; Wed, 24 Jun 2020 07:12:43 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 19/25] gitlab: convert jobs to use custom built containers Date: Wed, 24 Jun 2020 15:04:40 +0100 Message-Id: <20200624140446.15380-20-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x333.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Daniel P. Berrang=C3=A9 Now that we're building standard container images from dockerfiles in tests/docker/dockerfiles, we can convert the build jobs to use them. The key benefit of this is that a contributor can now more easily replicate the CI environment on their local machine. The container images are cached too, so we are not spending time waiting for the apt-get/dnf package installs to complete. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20200622153318.751107-4-berrange@redhat.com> [AJB: tweak naming convention] Signed-off-by: Alex Benn=C3=A9e --- .gitlab-ci.yml | 187 +++++++++++++++++++++---------------------------- 1 file changed, 81 insertions(+), 106 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 72d688875fd..a7abc55a5c6 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,133 +9,108 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' =20 -.update_apt_template: &before_script_apt - before_script: - - apt-get update -qq - - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make - genisoimage - - JOBS=3D$(expr $(nproc) + 1) - -.update_dnf_template: &before_script_dnf - before_script: - - dnf update -y - - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-dev= el - make python3 perl-podlators perl-Test-Harness pixman-devel zlib-de= vel - - JOBS=3D$(expr $(nproc) + 1) +.native_build_job_template: &native_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + before_script: + - JOBS=3D$(expr $(nproc) + 1) + script: + - mkdir build + - cd build + - if test -n "$TARGETS"; + then + ../configure --enable-werror $CONFIGURE_ARGS --target-list=3D"$TAR= GETS" ; + else + ../configure --enable-werror $CONFIGURE_ARGS ; + fi + - make -j"$JOBS" + - make -j"$JOBS" $MAKE_CHECK_ARGS =20 build-system1: - stage: build - image: ubuntu:19.10 - <<: *before_script_apt - script: - - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev - libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev libvdepl= ug-dev - - mkdir build - - cd build - - ../configure --enable-werror --target-list=3D"aarch64-softmmu alpha-sof= tmmu - cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-so= ftmmu - mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-soft= mmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-= softmmu + moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc= -softmmu + riscv64-softmmu sparc-softmmu + MAKE_CHECK_ARGS: check =20 build-system2: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y SDL2-devel libgcrypt-devel brlapi-devel libaio-devel - libfdt-devel lzo-devel librdmacm-devel libibverbs-devel libibumad-d= evel - libzstd-devel - - mkdir build - - cd build - - ../configure --enable-werror --target-list=3D"tricore-softmmu unicore32= -softmmu - microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-so= ftmmu - sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-sof= tmmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-sof= tmmu + riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-sof= tmmu + xtensa-softmmu nios2-softmmu or1k-softmmu + MAKE_CHECK_ARGS: check =20 build-disabled: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-rdma --disable-slirp --disable-c= url + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --disable-rdma --disable-slirp --disable-curl --disable-capstone --disable-live-block-migration --disable-glusterfs --disable-replication --disable-coroutine-pool --disable-smartcard --disable-guest-agent --disable-curses --disable-libxml2 --disable-t= pm --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user - --target-list=3D"i386-softmmu ppc64-softmmu mips64-softmmu i386-linu= x-user" - - make -j"$JOBS" - - make -j"$JOBS" check-qtest SPEED=3Dslow + TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + MAKE_CHECK_ARGS: check-qtest SPEED=3Dslow =20 build-tcg-disabled: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - dnf install -y clang gtk3-devel libusbx-devel libgcrypt-devel - - mkdir build - - cd build - - ../configure --cc=3Dclang --enable-werror --disable-tcg --audio-drv-lis= t=3D"" - - make -j"$JOBS" - - make check-unit - - make check-qapi-schema - - cd tests/qemu-iotests/ - - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 + <<: *native_build_job_definition + variables: + IMAGE: centos8 + script: + - mkdir build + - cd build + - ../configure --disable-tcg --audio-drv-list=3D"" + - make -j"$JOBS" + - make check-unit + - make check-qapi-schema + - cd tests/qemu-iotests/ + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033= 048 052 063 077 086 101 104 106 113 148 150 151 152 157 159 160 163 170 171 183 184 192 194 197 208 215 221 222 226 227 236 253 277 - - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 102 = 122 + - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 1= 02 122 124 132 139 142 144 145 151 152 155 157 165 194 196 197 200 202 208 209 215 216 218 222 227 234 246 247 248 250 254 255 257 258 260 261 262 263 264 270 272 273 277 279 =20 build-user: - stage: build - <<: *before_script_apt - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-system --disable-guest-agent - --disable-capstone --disable-slirp --disable-fdt - - make -j"$JOBS" - - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + CONFIGURE_ARGS: --disable-system --disable-guest-agent + --disable-capstone --disable-slirp --disable-fdt + MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-l= inux-user =20 build-clang: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y clang SDL2-devel libattr-devel libcap-ng-devel xfsprogs-= devel - libiscsi-devel libnfs-devel libseccomp-devel gnutls-devel librbd-de= vel - - mkdir build - - cd build - - ../configure --cc=3Dclang --cxx=3Dclang++ --enable-werror - --target-list=3D"alpha-softmmu arm-softmmu m68k-softmmu mips64-softm= mu - ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-us= er" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --cc=3Dclang --cxx=3Dclang++ + TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu + ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user + MAKE_CHECK_ARGS: check =20 build-tci: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - TARGETS=3D"aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86= _64" - - mkdir build - - cd build - - ../configure --enable-tcg-interpreter - --target-list=3D"$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; = done)" - - make -j"$JOBS" - - make run-tcg-tests-x86_64-softmmu - - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/px= e-test - - for tg in $TARGETS ; do - export QTEST_QEMU_BINARY=3D"${tg}-softmmu/qemu-system-${tg}" ; - ./tests/qtest/boot-serial-test || exit 1 ; - ./tests/qtest/cdrom-test || exit 1 ; - done - - QTEST_QEMU_BINARY=3D"x86_64-softmmu/qemu-system-x86_64" ./tests/qtest/p= xe-test - - QTEST_QEMU_BINARY=3D"s390x-softmmu/qemu-system-s390x" - ./tests/qtest/pxe-test -m slow + <<: *native_build_job_definition + variables: + IMAGE: fedora + script: + - TARGETS=3D"aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x = x86_64" + - mkdir build + - cd build + - ../configure --enable-tcg-interpreter + --target-list=3D"$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '= ; done)" + - make -j"$JOBS" + - make run-tcg-tests-x86_64-softmmu + - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest= /pxe-test + - for tg in $TARGETS ; do + export QTEST_QEMU_BINARY=3D"${tg}-softmmu/qemu-system-${tg}" ; + ./tests/qtest/boot-serial-test || exit 1 ; + ./tests/qtest/cdrom-test || exit 1 ; + done + - QTEST_QEMU_BINARY=3D"x86_64-softmmu/qemu-system-x86_64" ./tests/qtes= t/pxe-test + - QTEST_QEMU_BINARY=3D"s390x-softmmu/qemu-system-s390x" ./tests/qtest/= pxe-test -m slow --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008166; cv=none; d=zohomail.com; s=zohoarc; b=BvYzNzzRhLnO3p45wT9Ar3AjoF4BYT4y8sRrQ4fn7d3k9YtyXre6DgnqyKW3T6/5c6cwei56r1QWns9NTQquNQs1YInRV2UAqWwaI8V92GioYuSH7ZZgW0o+A3FmRzYWY8E7CiTkJf+eUVUQOC+nqco4iDOLdvZ9tnMHuI7L+D0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008166; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=WD6VNcX9MpLlUaAe843m6W+P0R0E6C+GHfBa0ZiWKL8=; b=kKhDVWZuqrw2FB/HKh365dLsvJPrnZCCE8QXw+TMMsltGs17ZG/SxufjmgSooZPoKGOlO8NkvF55J9Ezm5wXEzrhvIynUCPnvRZHZ7rl+OHz6pubkliNTjQjp4m9iMQc5356sVrZAPA/jG7IE0ji77e16t9eXd5MOyjR7ST7yOA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008166544347.44925652637664; Wed, 24 Jun 2020 07:16:06 -0700 (PDT) Received: from localhost ([::1]:47496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6C5-0001qw-H2 for importer@patchew.org; Wed, 24 Jun 2020 10:16:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33514) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68u-00047l-1q for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:48 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:40418) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68s-0005vN-BT for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:47 -0400 Received: by mail-wr1-x431.google.com with SMTP id h5so2443927wrc.7 for ; Wed, 24 Jun 2020 07:12:45 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id e4sm1020934wrt.97.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 59FE31FFA9; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WD6VNcX9MpLlUaAe843m6W+P0R0E6C+GHfBa0ZiWKL8=; b=wskmbiq6B0bDmhHqaq+JKx4A3ASBEDCd4VUB0g9n4AwmKCCjXDuy285gB64yqW6Jux TWfX08nwJNpDzaoFcHIq7HoVxEmWYlmj2zCq/Qa/CBGFB3DpJKjzJCMsPWAZwH9rSzAZ lfxBm6s1mLeNg20ch7zhUIehous6cLRbG/RwTCxC6L+b5qztarw/6I3LX3oCgInWHotT x0gYhiRDcHtfeyWtNoE1upryP7mfCiksNIrw/1+mqn6xYHtH9CuWnFOiWRZQLqiatZ+h MdH7714PQ9eLFwDPpc9Cufz3KavE0SYOLgJ7dEzKF88dTCdlShG32dc5dpVQRyKlX7F1 7s0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WD6VNcX9MpLlUaAe843m6W+P0R0E6C+GHfBa0ZiWKL8=; b=cPejsaQkjroCmx/qETIlb2Ut64pXVOX6XG0UzdX6287bLJRRLxoa8L4AfIx4gZU0YO 6yq5+9owQNTadz1xp5ClLVDBGXNtvsRA35IULaBy/j7gYMIEijErNU11NIPEsSQGLiht aar4yJ2BAw5tyqrcfqd5vLJrab6TppccKt2nakFNyGorcJjdgGWLBelstNwnEExu/nNd t1v5AqqybyPL7xVlk5IIHjNJqE9n56ewoO4bkVmkUg3x2ESmDH9GQUlHsxL7XIWgQR8D WT9e5RJT0mG0byj/+oVrqb79WCDHh48hRC8kd62Xu/ewmmdS7GgmannrZtxSkMm/kgnW JXhw== X-Gm-Message-State: AOAM531TFjKjPwxTP/N2B/1pf1SWH0j/INwskqox7lRaCiSOJgsSlTDf psmJkT1CxgU3DbJg4bpxrkg4mQ== X-Google-Smtp-Source: ABdhPJyEHlX+IbTTV/DIhFY44XCWufllIbmo8GApP7tKEoJJNZHCUbfzE6xH6H7oWYbKAUypt8SUSA== X-Received: by 2002:adf:81c8:: with SMTP id 66mr26929810wra.348.1593007964920; Wed, 24 Jun 2020 07:12:44 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 20/25] gitlab: build containers with buildkit and metadata Date: Wed, 24 Jun 2020 15:04:41 +0100 Message-Id: <20200624140446.15380-21-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x431.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) According to the documentation to be able to use --cache-from for remote registries you need to enable both buildkit and inline the metadata. We want to do this to support pulling from gitlab when users build their local docker images. Signed-off-by: Alex Benn=C3=A9e --- .gitlab-ci.d/containers.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index 580439647ef..ba9c24e98f1 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -13,7 +13,9 @@ script: - docker pull "$TAG" || docker pull "$COMMON_TAG" || true - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker= /dockerfiles/$NAME.docker - - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$= TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - DOCKER_BUILDKIT=3D1 docker build --cache-from "$TAG" --cache-from "$= COMMON_TAG" --tag "$TAG" + --build-arg BUILDKIT_INLINE_CACHE=3D1 + -f "tests/docker/dockerfiles/$NAME.do= cker" tests/docker/dockerfiles - docker push "$TAG" after_script: - docker logout --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008363; cv=none; d=zohomail.com; s=zohoarc; b=lDSQ+3bdTx5MsoPXcJNaLfGP/GeWOY2h5XefLR1W7vOHJQtMthJwP0FEUZDtP1vI3qJAmGhLgFoohnK8cOKvEnWI2NQJeAjOdM5oolTHiYmFkMT+TaTNPYuJFUTKv2ELRFChBK0IeD4k7V5O15rcM2/Y8zwJ62uUpA2lLbI72U4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008363; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2kkJsCsmjZ6u2MwSGAsWWjkvARamWfB9kpoiOtV7oLc=; b=B2BKszIOyabcRSVO8ork4IvGYW7C+yZ55qhdb1QEG/eZ/y+kBiGA4C0RE2SepW6/EtpwEdk4vB+vdKCH/vonLtTSa9MpewxSXqDmSV+Och/IA2ZxsGZaqJPO9JyvWqk7oziiDmYJ3Lk7v1DM26J3uCt5SG0NBtZSoLrS4Ug0CnI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008363955709.2539168296167; Wed, 24 Jun 2020 07:19:23 -0700 (PDT) Received: from localhost ([::1]:33320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6FG-0007yt-JK for importer@patchew.org; Wed, 24 Jun 2020 10:19:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59404) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61c-0005NH-3M for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:16 -0400 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:54120) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61Z-0003Yc-Pu for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:15 -0400 Received: by mail-wm1-x343.google.com with SMTP id j18so2441419wmi.3 for ; Wed, 24 Jun 2020 07:05:13 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id r1sm18997755wrt.73.2020.06.24.07.04.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:04 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 707F51FFAA; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2kkJsCsmjZ6u2MwSGAsWWjkvARamWfB9kpoiOtV7oLc=; b=hyyFrqkwGkLUP6+PLih+TsKY3F1hRXCmQnL3uNwg+PEUaF6E3v9AjGMEkY9lZPtp2a 8Gdw9waGFQfkVBvOMxd9ruWCt88vlCIMcUnSkcheK0weGnBzp8ZVQJvwphKInmH1eGTa 19vXG7TrgTbRcsuuETPhHYqXad+A1N+eIRnTH8h6UzD+ZYDK9TyDjuj8TjLIt2HEUHup 1l/QWIoD86xLuP9WF7cj02bXwiVzPNClhwDFFUDY3ETJ5xiwVq2G8f2XgxI0bwTsUxDk l1oiPnoEmt5t7QnhS9VijNQSdNe0RJ0DGm7pOfM4cJzmwc1Nygch87UGKZI0Z01pm9Ya vgcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2kkJsCsmjZ6u2MwSGAsWWjkvARamWfB9kpoiOtV7oLc=; b=DjGQn6wZtB2zcP9LDkiyHTJB+vsQGz2D0HcAv18a+n05iTdJkcm+h6zyip0FxSIwsV r6M+SLITKUjwnGtfzoD3Tq+kSvxQtK6ZMbP3Mplb0daRfFeyWyMykPVzfptUmjTo93r+ RU9wCbATrWlSHfWjOPjpEvUOaf0UxzDskhEgZ3J8NgtbVjR/ygICV8vkges73UHa2qT2 YmpEdX4ajIwKDLNPHxFLrNcsdELpa/3BJehjIQLFtjzKx6b7xwaCl9LUIy48cKA8FY3G R+IHlTkGoiv3SnfLJ3LCh9CAUbnzkjCVOLUIzqEkC8kTfNFANdOkm6rvvqiygBRpExIQ ryDw== X-Gm-Message-State: AOAM533ASOKrWcaiZT7sayQLylrgZO9Om/t/414/A2yDNCnjDCkcCtOs Nxa1PjSwA5yxb3CNpKpmszRqDg== X-Google-Smtp-Source: ABdhPJwRBBTtjZgt0tLwhGppgBW+fPaGwhlUnFB9SHuqwSWt0XJB9tnKM5KWlSvQOZg+QkHHb8s/2Q== X-Received: by 2002:a7b:c926:: with SMTP id h6mr27226435wml.184.1593007511369; Wed, 24 Jun 2020 07:05:11 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 21/25] tests/docker: add --registry support to tooling Date: Wed, 24 Jun 2020 15:04:42 +0100 Message-Id: <20200624140446.15380-22-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::343; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x343.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This allows us to point the tools towards a registry from which they can grab pre-built layers instead of doing everything from scratch each time. To enable this we need to be using the DOCKER_BUILDKIT engine. [AJB: note registry.gitlab.com/stsquad/qemu is for my testing, the final version will see DOCKER_REGISTRY default to registry.gitlab.com/qemu-= project/qemu] Signed-off-by: Alex Benn=C3=A9e --- tests/docker/Makefile.include | 3 +++ tests/docker/docker.py | 25 +++++++++++++++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 7bac1a67e3e..4c5ebc6197f 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -13,6 +13,7 @@ DOCKER_IMAGES :=3D $(sort $(notdir $(basename $(wildcard = $(DOCKER_FILES_DIR)/*.doc DOCKER_TARGETS :=3D $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds DOCKER_CCACHE_DIR :=3D $$HOME/.cache/qemu-docker-ccache +DOCKER_REGISTRY :=3D $(if $(REGISTRY),$(REGISTRY),registry.gitlab.com/stsq= uad/qemu) =20 DOCKER_TESTS :=3D $(notdir $(shell \ find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f)) @@ -57,6 +58,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command,\ $(DOCKER_SCRIPT) build -t qemu/$* -f $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ + $(if $(DOCKER_REGISTRY),--registry $(DOCKER_REGISTRY)) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ $(if $(EXECUTABLE),--include-executable=3D$(EXECUTABLE)),\ @@ -213,6 +215,7 @@ endif @echo ' Include extra files in image.' @echo ' ENGINE=3Dauto/docker/podman' @echo ' Specify which container engine to run.' + @echo ' REGISTRY=3Durl Cache builds from registry (default:$(D= OCKER_REGISTRY))' =20 # This rule if for directly running against an arbitrary docker target. # It is called by the expanded docker targets (e.g. make diff --git a/tests/docker/docker.py b/tests/docker/docker.py index cc6f76caa60..09dedbf83f9 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -221,6 +221,8 @@ class Docker(object): """ Running Docker commands """ def __init__(self): self._command =3D _guess_engine_command() + if "docker" in self._command: + os.environ["DOCKER_BUILDKIT"] =3D "1" self._instance =3D None atexit.register(self._kill_instances) signal.signal(signal.SIGTERM, self._kill_instances) @@ -289,10 +291,16 @@ class Docker(object): return labels.get("com.qemu.dockerfile-checksum", "") =20 def build_image(self, tag, docker_dir, dockerfile, - quiet=3DTrue, user=3DFalse, argv=3DNone, extra_files_c= ksum=3D[]): + quiet=3DTrue, user=3DFalse, argv=3DNone, registry=3DNo= ne, + extra_files_cksum=3D[]): if argv is None: argv =3D [] =20 + if registry is not None: + dockerfile =3D dockerfile.replace("FROM qemu/", + "FROM %s/qemu/" % + (registry)) + tmp_df =3D tempfile.NamedTemporaryFile(mode=3D"w+t", encoding=3D'utf-8', dir=3Ddocker_dir, suffix=3D".= docker") @@ -313,8 +321,14 @@ class Docker(object): =20 tmp_df.flush() =20 - self._do_check(["build", "-t", tag, "-f", tmp_df.name] + argv + - [docker_dir], + build_args =3D ["build", "-t", tag, "-f", tmp_df.name] + if registry is not None: + cache =3D "%s/%s" % (registry, tag) + build_args +=3D ["--cache-from", cache] + build_args +=3D argv + build_args +=3D [docker_dir] + + self._do_check(build_args, quiet=3Dquiet) =20 def update_image(self, tag, tarball, quiet=3DTrue): @@ -403,6 +417,8 @@ class BuildCommand(SubCommand): parser.add_argument("--add-current-user", "-u", dest=3D"user", action=3D"store_true", help=3D"Add the current user to image's passwd= ") + parser.add_argument("--registry", "-r", + help=3D"cache from docker registry") parser.add_argument("-t", dest=3D"tag", help=3D"Image Tag") parser.add_argument("-f", dest=3D"dockerfile", @@ -458,7 +474,8 @@ class BuildCommand(SubCommand): for k, v in os.environ.items() if k.lower() in FILTERED_ENV_NAMES] dkr.build_image(tag, docker_dir, dockerfile, - quiet=3Dargs.quiet, user=3Dargs.user, argv=3Da= rgv, + quiet=3Dargs.quiet, user=3Dargs.user, + argv=3Dargv, registry=3Dargs.registry, extra_files_cksum=3Dcksum) =20 rmtree(docker_dir) --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008222; cv=none; d=zohomail.com; s=zohoarc; b=hVX7pHYRcd9ZlZZe7orLf1G+bF0Aq9JJUoGrPPHfAw20T0u86YrZ8ueRDKZ5C5qLsjWGvslFm3NFZ6YD8ySwe4aUjKdfIAHbGyMXiHeUOsw+OG2EEXkxSwGDcne+pCBrwncTngM/vN/jHB1K5Rr8AQx5fWs1b6Ele4OKY57WB/4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008222; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4JsqTbe4nx5Y4ze+RsCvycmftPXZdnpOMBPVnwlH7Pw=; b=QHqZZJAVpdjkAD20rnxhyP/nYBCsI+5oVM7pJaJGunO7PlYCJOspA72A1JyvRH2CGEvgJjOgNh76/T255B3qnsh1xtvWpNymZODQHNfbw3z3kiq3oXoq1vq28dP73juySlzlmGnuZX7uKrQUAtmdtsgl4UTkorcuTSwb3xDa+SY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008222478123.62983339743073; Wed, 24 Jun 2020 07:17:02 -0700 (PDT) Received: from localhost ([::1]:50672 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Cz-00038f-EN for importer@patchew.org; Wed, 24 Jun 2020 10:17:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68y-0004I3-E7 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:52 -0400 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:51284) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68w-0005xn-C4 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:52 -0400 Received: by mail-wm1-x332.google.com with SMTP id 22so2474749wmg.1 for ; Wed, 24 Jun 2020 07:12:49 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z16sm17596847wrr.35.2020.06.24.07.12.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 8804C1FFAB; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4JsqTbe4nx5Y4ze+RsCvycmftPXZdnpOMBPVnwlH7Pw=; b=Zy5uGhZbVrUE8s9R5MwQljmeTArI2h6EuIR7XdyOUQB2h42SYTpt+fTHzwLIAxx9gA RazQPygD9QjKE8ReAH+2lo/+RcbihCXmbhIIzeNMP8KgzB9AcY3SHqGY+rASAmlDf8So 3trIBocAW6fjoDQIq3FlYX3YGCu+aHrJYMaK1h3RlIT2CNiEfDt9FRKwVRDlZrhODdai w6THDSkse2iaMdF9IyIpDI0YTA0GwZMpUlTk8tQvFzv+UPh/TYmFGyEjISETJ4hiwgtL vZ2OkrYWvURyseBtKoAOFjdCMRfuu1QuZFiR27/hW8uhYvzPqdilDyXCOB0gATO5dInG +8lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4JsqTbe4nx5Y4ze+RsCvycmftPXZdnpOMBPVnwlH7Pw=; b=YJBFVT6ZrGeTQ11uadxWw+q0N9PY25TB7Rxwi7w/1xXzg8+UX2T42fOQaJZT49LZpy VKB/NFUBra4gg/1haHsv31qjg3WWeLHwyCc8NVLOjTiLqZinUy+9fLZpLDUMMLO+adNN BaMEEDMoqIn/K1JTnJd8c0u74sQOBy82Q0N2D34rn5XmBvgK1oziDy72/M9WosEOTuJp vX3mOkNvvFiQELTepo5icp9NYsBp1l6GOQvfoD9JfIiKikq1bLqiIM1KgYgbThMup0vw RcaEc/w+skA5pzdXXHEAyJbPN5cjdqgEbh5ind1vwex4Yvk+c2jKiXk0sA69HaVandhi dMKA== X-Gm-Message-State: AOAM532Lho1lF3XFm9CxquzU/i4OiAfqSqQbtB1m7ZPWCQGZs8bmcyOI pvOYATF2j2666tm7JTRPJ5GIIQ== X-Google-Smtp-Source: ABdhPJwpIwx4oujo6JpFWsB6StuWkOhS1lIrCOl4Yre6INAU6VAQJK7bnBTIAGoaoaQq9+x9U0y1TA== X-Received: by 2002:a1c:1d04:: with SMTP id d4mr31544499wmd.156.1593007968819; Wed, 24 Jun 2020 07:12:48 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 22/25] gitlab: add acceptance testing to system builds Date: Wed, 24 Jun 2020 15:04:43 +0100 Message-Id: <20200624140446.15380-23-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x332.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) As part of migrating things from Travis to GitLab add the acceptance tests. To do this: - rename system1 to system-ubuntu-main - rename system2 to system-fedora-misc - split into build/check/acceptance - remove -j from check stages - use artifacts to save build stage - add post acceptance template and use Signed-off-by: Alex Benn=C3=A9e Message-Id: <20200622143204.12921-16-alex.bennee@linaro.org> --- v2 - updated with danp's docker changes - use needs instead of dependancies --- .gitlab-ci.yml | 65 +++++++++++++++++++++++++++++++++++++++++++++++--- .travis.yml | 23 ------------------ 2 files changed, 62 insertions(+), 26 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a7abc55a5c6..7fcbdacd9e9 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,8 +1,12 @@ +# Currently we have two build stages after our containers are built: +# - build (for traditional build and test or first stage build) +# - test (for test stages, using build artefacts from a build stage) stages: - containers - containers-layer2 - containers-layer3 - build + - test =20 include: - local: '/.gitlab-ci.d/edk2.yml' @@ -24,26 +28,81 @@ include: ../configure --enable-werror $CONFIGURE_ARGS ; fi - make -j"$JOBS" - - make -j"$JOBS" $MAKE_CHECK_ARGS + - if test -n "$MAKE_CHECK_ARGS"; + then + make $MAKE_CHECK_ARGS ; + fi + +.native_test_job_template: &native_test_job_definition + stage: test + image: $CI_REGISTRY_IMAGE/$IMAGE:latest + script: + - cd build + - make $MAKE_CHECK_ARGS =20 -build-system1: +.post_acceptance_template: &post_acceptance + after_script: + - python3 -c 'import json; r =3D json.load(open("tests/results/latest/= results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] no= t in ("PASS", "SKIP")]' | xargs cat + - du -chs $HOME/avocado/data/cache + +build:system-ubuntu-main: <<: *native_build_job_definition variables: IMAGE: ubuntu2004 TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-= softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc= -softmmu riscv64-softmmu sparc-softmmu + artifacts: + paths: + - build + +check:system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build:system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check =20 -build-system2: +acceptance:system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build:system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 + MAKE_CHECK_ARGS: check-acceptance + +build:system-fedora-alt: <<: *native_build_job_definition variables: IMAGE: fedora TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-sof= tmmu riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-sof= tmmu xtensa-softmmu nios2-softmmu or1k-softmmu + artifacts: + paths: + - build + +check:system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build:system-fedora-alt + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check =20 +check:system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build:system-fedora-alt + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + build-disabled: <<: *native_build_job_definition variables: diff --git a/.travis.yml b/.travis.yml index 74158f741b1..c24dfbe377f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -289,29 +289,6 @@ jobs: python: 3.6 =20 =20 - # Acceptance (Functional) tests - - name: "GCC check-acceptance" - dist: bionic - env: - - CONFIG=3D"--enable-tools --target-list=3Daarch64-softmmu,alpha-s= oftmmu,arm-softmmu,m68k-softmmu,microblaze-softmmu,mips-softmmu,mips64el-so= ftmmu,nios2-softmmu,or1k-softmmu,ppc-softmmu,ppc64-softmmu,s390x-softmmu,sh= 4-softmmu,sparc-softmmu,x86_64-softmmu,xtensa-softmmu" - - TEST_CMD=3D"make check-acceptance" - - CACHE_NAME=3D"${TRAVIS_BRANCH}-linux-gcc-acceptance" - after_script: - - python3 -c 'import json; r =3D json.load(open("tests/results/lat= est/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"= ] not in ("PASS", "SKIP")]' | xargs cat - - du -chs $HOME/avocado/data/cache - addons: - apt: - packages: - - python3-pil - - python3-pip - - python3-numpy - - python3-opencv - - python3-venv - - rpm2cpio - - tesseract-ocr - - tesseract-ocr-eng - - # Using newer GCC with sanitizers - name: "GCC9 with sanitizers (softmmu)" addons: --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008056; cv=none; d=zohomail.com; s=zohoarc; b=YpMmdWV9q9NCrpkE2Z36UbLW9UT/kjfzSMRUuwwKxp4WJS3f8E4ThnAoKYjt4mqd2RejpCIEZnjaBN8dqRbhMjoWBYjOhVhOBKV/zrY0wxQrvkYY4/m1+89E9AOx1Gjc8yS1mYLuxUegW0WriCXD7BwyhXtIPhNWDiOlIVO3PSI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008056; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OTrZxr+gr7TRXS20xbwh/wb3Pjs8p21fdYjxEPsPxt4=; b=nnsF54ker6LRz2fQbYzxxpUo9SDcdP6ud+B8t/R6VVOqn8NcGo6WP9OvOgHSAAnk4LZIIzK/DGU/MBK2uR8lcLnPHUs6HrvTBhJV3o39FJgyDz19o5pz3GEGd1vv8+OoGlBCtGS+xs/lEYOZSVkNlwbSRbbbMsFZX8s91zkUHME= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008056082873.5115735993513; Wed, 24 Jun 2020 07:14:16 -0700 (PDT) Received: from localhost ([::1]:39252 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6AJ-0005ul-55 for importer@patchew.org; Wed, 24 Jun 2020 10:14:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33478) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68s-000459-Fl for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:46 -0400 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]:50429) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68q-0005ui-P9 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:46 -0400 Received: by mail-wm1-x336.google.com with SMTP id l17so2482674wmj.0 for ; Wed, 24 Jun 2020 07:12:44 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id s5sm8358484wmj.18.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 9EB621FFAC; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OTrZxr+gr7TRXS20xbwh/wb3Pjs8p21fdYjxEPsPxt4=; b=nW6IQeizoebZzBBnUvlFkI1djmJVrNzyP9KYRi1BhllJgQsA31/tFl6+Qeq44yCpli RjUN7LkGuoYegmsqR3LyEJQlp/lRkyMqp6L44TpAdymaueX/+FTxPrGUPrbr7CMCLInR 3IXGibs2Z6tOvvYRMzriOwI77c0kMHZPTqf0CSVoQDCun5PeBTq4k0L+Z6dGe9SGlffz x1yN5T+REERB1tQWSos3GrjsDBx0+JVfc6FY9gsIX1IgQcULu/YMNWyMJQBnLtCnm8uV OElPuWU7e8z/fcKxfbSVYHzm23Q7RAoj6H1ZTqYMU8TpTtOsFKXOgZus9af+Lob8yYm3 pMfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OTrZxr+gr7TRXS20xbwh/wb3Pjs8p21fdYjxEPsPxt4=; b=YOZlFoY050RetW0RCy3o8bdBuVoQ4KQoRPJwdk33cRf/do3Oo5nvD12WNiiKfnGrqU XsbWTdiEXJ6PB11YPP2CwKLEa/n5wimcNHfPHQmZvtdckO8nshCN8bnwZTq09EYpwX+K cqg/h3k1sGVUKEJD4n3lB8B7ZrLvnBSj7GpDrG2dA2ri7+HhTn+gQa8jnvKngse6uvnn rA1+Ek+eM1pbHHWQ4VlIAH7jdzrJJHDnEW5jGWn26ldFZmDM7z2qpq5RQPVld+ympfTx 8tuaW8kwOZyvtnnOFWd5SpHVct9s3emfA27SIFEGK5Yp1oPbpgOG+wggAfgA2C3B+KCh Cqlg== X-Gm-Message-State: AOAM530kbjG1BtK/oM9B8dlsXoRHGwOb/AUoh+TcBn7JMaK7+mTByBuD J/Wk/doWsQqBgkGYG3Q3/vq0kg== X-Google-Smtp-Source: ABdhPJzM7WVDf5Bkf19BisW2Q4yQNCqdg/bDj6kZ9ilAHQW0fvNyyDGxF3K8PVB70bvAC/6x9Pqz4g== X-Received: by 2002:a1c:c90a:: with SMTP id f10mr17217993wmb.121.1593007962417; Wed, 24 Jun 2020 07:12:42 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 23/25] gitlab: enable check-tcg for linux-user tests Date: Wed, 24 Jun 2020 15:04:44 +0100 Message-Id: <20200624140446.15380-24-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::336; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x336.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) We should have the containers available now. Signed-off-by: Alex Benn=C3=A9e --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7fcbdacd9e9..54da29dd384 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -142,7 +142,7 @@ build-user: IMAGE: ubuntu2004 CONFIGURE_ARGS: --disable-system --disable-guest-agent --disable-capstone --disable-slirp --disable-fdt - MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-l= inux-user + MAKE_CHECK_ARGS: check-tcg =20 build-clang: <<: *native_build_job_definition --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008053; cv=none; d=zohomail.com; s=zohoarc; b=LEXQDO7oNWw56LStxxGSXhgwriF7OE972hMOY4Fp5teMDF74jt3QVtzZbRnrwfOaEdJQL1EF5RDfDkDXLfC4zk6yl1dCA1t2vlyLde0/3F2eypW2f89SoAF3amM3cCwqlQv0kwi7xGY8ywFE4WgSTHktjY1dvZ0jAVQJ+Tjb5Bw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008053; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kkJ3TlSVKMVotvh/AON95M2rOXL6DLcXLqurDZ1ML4w=; b=dV924EjWZbSqS/A56lKV0e6+oJBcpWmnX7ILuYklXmy3UQgIaXcn7bq1T80RhXkh//p7qZuJfMi/6N9/R5EGpEldcYS1r9A5vh3tlPJns82nt85ucuypqq2tf/RLgslk1z6yDkuU0Dz5USSWDz69K1IbxLsSZ0fwrodG1TyGFNI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008053821413.80191192827067; Wed, 24 Jun 2020 07:14:13 -0700 (PDT) Received: from localhost ([::1]:39036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6AG-0005nb-Pw for importer@patchew.org; Wed, 24 Jun 2020 10:14:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68r-00043l-6d for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:45 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:39624) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68p-0005ta-B3 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:44 -0400 Received: by mail-wm1-x32b.google.com with SMTP id t194so2677415wmt.4 for ; Wed, 24 Jun 2020 07:12:42 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n16sm20955950wrq.39.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id B500D1FFAE; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kkJ3TlSVKMVotvh/AON95M2rOXL6DLcXLqurDZ1ML4w=; b=U5+yr8Ah+wdD1Tv5Yq6iMv4KxqWlivSYl+AqLVsBoTZfVMuIeoharISrhSuMY0aEXJ 0dnXh71iWsL/5A/dvwAQO1uElLHONHgcCnBtRTzavwMKuWQcZi04pT/alFvPHtRoOlDR /Q4bqaaxfhG3F9uZJ/BqYB3tFvLqTTRYQywLAV/nxs5RYG6AgREsT9sZOPuKL5pZxnhO qJFb5MSSMN5G5s4GhWN3QHzwX0XgWdgtGGnl3G5FMJhZWVQ3eJk43gGA5NR358GmP6nE LkQkNp62ARw1R8TJ/SrocCkCaLzNEtMg2JFPRrmbN46rbV10xyjyqI/1KppP5qVnz7YX MTcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kkJ3TlSVKMVotvh/AON95M2rOXL6DLcXLqurDZ1ML4w=; b=HjIEsCoVKSCQZLP+7R+YPok806vxdv5fsT/782+CN3I2XBJLz+Fv2o1JMbf7PmYswM zzqryswfBbIaDmOcA8kUiRq3VG275Bb4iqBN3/XzvRJxaqn4KHbncxaeiBcbfRROjSOg IZJDFUwrAEbEcy9uYYjH8HjYyoDbKIqTuMUYv+Z1kf4U8OGyOoXrHaMICCsL1APCHRjR MNiyT7a2AFRlq5Sl0ULbUrhlowvpArHk7Cygi21kdfW1uS82SGjy6lOegIF5atuvrb7D DvnM8h/H6PKRG77E6MgTHhOxS6ToEm6X/6f3kM6T0O5uYKYZIIBlQsRHYTa8cKF0tUP8 p5tw== X-Gm-Message-State: AOAM532aJPPEoVO6eUrnsNwQV0hvOy/n5Bm7Aa+E2w7jxycNSJ0j0MM5 FTXhmBif8B9VhhmEDnjRgNuQew== X-Google-Smtp-Source: ABdhPJzMD1grAq3rQlpuGeMqcTz0AwjjIqo5rMy+rdQh1z6jDAA79a5G2HCymUqOLZViJeGZQJD0KQ== X-Received: by 2002:a05:600c:2144:: with SMTP id v4mr19647966wml.128.1593007961346; Wed, 24 Jun 2020 07:12:41 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 24/25] gitlab: add avocado asset caching Date: Wed, 24 Jun 2020 15:04:45 +0100 Message-Id: <20200624140446.15380-25-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) These can be quite big so lets cache them. I couldn't find any nots on ccache in the gitlab docs so I've just ignored it for now. Signed-off-by: Alex Benn=C3=A9e Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-Id: <20200622143204.12921-19-alex.bennee@linaro.org> --- .gitlab-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 54da29dd384..683a1d90fe6 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -8,6 +8,12 @@ stages: - build - test =20 +# We assume GitLab has it's own caching set up for RPM/APT repositories so= we +# just take care of avocado assets here. +cache: + paths: + - $HOME/avocado/data/cache + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' --=20 2.20.1 From nobody Tue May 7 23:40:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1593008403; cv=none; d=zohomail.com; s=zohoarc; b=e6tutS1N9gJbnkc0TZAwmb1dU1UsKZhX218YOuunaxX4+oInjtyoeGjpuGMJEKk0ehhcAtFkdIZz4kV0XR8XJkNabr6W6vgW356wiIi5P50NAkEbFKZssb+Yjum5uyEj+c/yx27x7E4PNU0yYR5ZjRVYvYEmnW3craPzu0FCtxQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593008403; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NSbE/oPaUPqzCfEUFHpvKz9daAgMJApN4FMtaAwBhjs=; b=CQuMKbXPYy5epWLxc6oh1Ln0DdJubC3f88y5Gp/qkrWnxXeNLo6hQ6d8lndYExZw0x8EeYy4JO5xf3/A6h82sU+8134QyrCJj/j/Ck9sUa98KVwRWP0sWrHKbggX6NipY1KjCKwZpiWjsh+zdIRtpL5mbi8PCWiYeJ8TK+imvEQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1593008403774759.9667142653178; Wed, 24 Jun 2020 07:20:03 -0700 (PDT) Received: from localhost ([::1]:35064 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Fu-0000ZE-33 for importer@patchew.org; Wed, 24 Jun 2020 10:20:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68x-0004Ec-1Z for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:51 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:37848) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68v-0005wr-8M for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:50 -0400 Received: by mail-wr1-x436.google.com with SMTP id a6so2457966wrm.4 for ; Wed, 24 Jun 2020 07:12:48 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id u186sm8568589wmu.10.2020.06.24.07.12.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id CAFB01FFAF; Wed, 24 Jun 2020 15:04:48 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NSbE/oPaUPqzCfEUFHpvKz9daAgMJApN4FMtaAwBhjs=; b=xf4HwYVQa2TwFiK7FK4SfJ7rbA0WfBzMSdNwuq9AtPmuET4wwmYI11fqOhMzGhgZbT nhn2iGl4RX6CU4bLiWV6+BJ3XacG8upCIgnVe8Z3vP1ZnLXT8mMkgFUa4ospWZw30V0v Y3rtZ1hlfIywtrzWgaKQeqE03iZmpmMho5ayD2Uuh9sKoCPgFa+pXS8xMcg51CJxsfXF cKFpop+2G5uf8VfKrK5rkOjRi7FnSaH1yJgweyH46nH9kQCiZsvU5Mgdms7xCKlrMpew cxu8Y8G473nSrJwfhodkRgUwJn2n4nHv0IyYtuc5ElAqFqsT5NrCCU2WdSfFw/e3OiBU 5LtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NSbE/oPaUPqzCfEUFHpvKz9daAgMJApN4FMtaAwBhjs=; b=O6GiJ73TINB5rR6cN/CcbAfURRZmTrppPnjCuF4dMKsYkO5yDOznUxflejpIdBEgSM aHojChJ+JMSxlOGvs6UZOLCrVmM4yNbh6cGIuevC3RgY1d9GfHUr6i98Ys7LP58XXr7l Rw+spELqeT/A/foA6R1tVpDMdI1VY5C8/VoQKqv5NfPf8pDf4O51okJlF/eU6LgRULSL kEXb8gRF1hQ6xzMkEwj42nuv1cv+Gcxggthvu2VHB5lPm6hvMYr/YYFdXE0bax0LpJSd cH/ibkaqGeGiVLx5hrh3gT1A+zxO5zSdWb0jKukF8sAIJBYQLZsSherofKLESIGhpOeZ MnIg== X-Gm-Message-State: AOAM531P5DqHwTDU4ZpcJ89UbZtlKbV4KYOHs0OxcYbSb42n3r5iTtuK jmPTDK6bElI5FY8sjt5wnxqMVA== X-Google-Smtp-Source: ABdhPJwxt6p3HNem1VIJO66ghucbwjgASoziL3+xzwdDbpJZjNmtj0qNm7Or4gZAUuJ9GkM80im6Og== X-Received: by 2002:adf:f14c:: with SMTP id y12mr30961671wro.30.1593007967876; Wed, 24 Jun 2020 07:12:47 -0700 (PDT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 25/25] gitlab: split build-disabled into two phases Date: Wed, 24 Jun 2020 15:04:46 +0100 Message-Id: <20200624140446.15380-26-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x436.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) As we run check-qtest in "SLOW" mode this can timeout so split into two jobs. Signed-off-by: Alex Benn=C3=A9e --- .gitlab-ci.yml | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 683a1d90fe6..2567c88f99d 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -109,7 +109,7 @@ check:system-fedora-alt: IMAGE: fedora MAKE_CHECK_ARGS: check-acceptance =20 -build-disabled: +build:system-fedora-disabled: <<: *native_build_job_definition variables: IMAGE: fedora @@ -120,6 +120,17 @@ build-disabled: --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + artifacts: + paths: + - build + +qtest:system-fedora-disabled: + <<: *native_test_job_definition + needs: + - job: build:system-fedora-disabled + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check-qtest SPEED=3Dslow =20 build-tcg-disabled: --=20 2.20.1