From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294587; cv=none; d=zohomail.com; s=zohoarc; b=aqOv2TG1rtzsEAOFTMDSLRejhHmrkR9QoT7dOKCsQhd0U11VyCb+VxklZC43QO2qjF2lB1yQZpp+Sgpzc1vjvzc3C4Qv+ejiL/zmyQhZ3AlWs4MFPkuC/KcHtFFGSJwb1xK5nOdfVFyDBa4014mxAM9clFTHgnKt2vB5VXSw+VI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294587; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=V5R842WAHGDJw5CPqYEdxqUrF1hwgdaC5kM/Nz8nv2o=; b=Jo2UrXmoZLI1Jgnw24+EbHDA9fTcsCLHdGBmWdjI/VJnahx4EJlNQhuKn8gv2O4XzeNbS4gmpKEspKnmN1XTR9XUquZwjIx5yjGLchQSHLT+oRq46Mq8BpUFahW7Vfn0xy/bGm61IDkioVH5ECYCiNeLdjkDVIX+sf16Cqfj0qM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294587074205.6041881411578; Tue, 12 May 2020 07:43:07 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-231-TVP0-QL9NZavnEaWjN6DtQ-1; Tue, 12 May 2020 10:42:44 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C8612835BB8; Tue, 12 May 2020 14:42:36 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AA6D462A28; Tue, 12 May 2020 14:42:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7BF0F4CAAD; Tue, 12 May 2020 14:42:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgRuR020142 for ; Tue, 12 May 2020 10:42:27 -0400 Received: by smtp.corp.redhat.com (Postfix) id 83FE95C1C3; Tue, 12 May 2020 14:42:27 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F7665C1B5; Tue, 12 May 2020 14:42:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294585; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=V5R842WAHGDJw5CPqYEdxqUrF1hwgdaC5kM/Nz8nv2o=; b=QkLfotaJoEtSiu4nJzuPENfya+Dbvdr9NVnBuzsig083JsFVY680d4PPJ7NN4SMgX/gGGN +VVrczvK8WLVtrd+VKKw4mSyrsQZ01oM0r3cpERSMHpi4CJ7dTnFa+tQ2Tlr9gHsti0MNH XXRH42Zt8hJ5o/3AzZEQnn7In3Su6rk= X-MC-Unique: TVP0-QL9NZavnEaWjN6DtQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 01/13] requirements: Introduce a requirements.txt file Date: Tue, 12 May 2020 16:42:07 +0200 Message-Id: <6b19225c261caba9d96e46fd7c8f3ca813a20b74.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Right now, lcitool's users have 2 options to satisfy the tool's dependencies: install them through the system package manager system-wide or install through pip manually. Since pip gives us the option to automate this process a tiny bit, let's ship a requirements file. This targets users who are used to work with Python virtual environments and install whatever it is necessary inside. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/requirements.txt | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 guests/requirements.txt diff --git a/guests/requirements.txt b/guests/requirements.txt new file mode 100644 index 0000000..900cfd6 --- /dev/null +++ b/guests/requirements.txt @@ -0,0 +1,2 @@ +ansible +jenkins-job-builder --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294585; cv=none; d=zohomail.com; s=zohoarc; b=CrgIIxCOUmQe+FTfOs7r9RUxfe+3cX1TVzRUKqokBTLWIUcWpP/ef7f/y1EgcuBeeBv6qzEvYInSZFDKwcpH0FABl+oTlHnTy0v3B0MrX51+sk+srj78SZ9GL9dKiysIFGa2rhA+qV8/8052pllglTqs0d4p4SSin0SGlYTrzIc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294585; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=13mK8RUQ47tMzFeRUvfX+g/HPS/qmR1QQO3lKTlvoLM=; b=d/HTIFImfQ2I+nrebpPYiKIeAhsnAGvQxxwMXBXSj5u99tLDX6odeYWEbFeRmHzcDM74FtESKpKVjIQ4XHGH7Q2oTsPyRgyToeVaWEgJxytXHBf4nHp3eXlav9IuDLGBwTSRYaG3w908RleKHn1ptabZRFgnEcS97UmLUggE+nY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1589294585001988.3813576224621; Tue, 12 May 2020 07:43:05 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-416-rJ3LeXQbOgmb9mm4G_QCwA-1; Tue, 12 May 2020 10:42:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2C2051B18BC6; Tue, 12 May 2020 14:42:39 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0AD465D9DD; Tue, 12 May 2020 14:42:39 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C73211809557; Tue, 12 May 2020 14:42:38 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgSUR020151 for ; Tue, 12 May 2020 10:42:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id BE2C45C1BB; Tue, 12 May 2020 14:42:28 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id D83905C1C3; Tue, 12 May 2020 14:42:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294583; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=13mK8RUQ47tMzFeRUvfX+g/HPS/qmR1QQO3lKTlvoLM=; b=gYvPEA26IXANS1o2Nb3GOeip/vBqaG9NZnlGIOW+3vd6RVWxmhhAmEJSO7GfA/vz+ak0h6 G1MhLRe7PO5zGh92p4T18fWcaTfXsE8xFidgbn2KHXqtEmhpXeC9FwgaJkcp175ZpyE7Is QVYZpHCtmzcy2fg7jnyLDNPzpJZE+q0= X-MC-Unique: rJ3LeXQbOgmb9mm4G_QCwA-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 02/13] lcitool: Decrease the indent when creating a tempdir for initrd injection Date: Tue, 12 May 2020 16:42:08 +0200 Message-Id: <241b440b9026636ac5a0e99aaf7bb906dd21f71a.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" The 'with' statement doesn't define a new code block [1], thus no local namespace is created. Therefore, we can still access the @content variable outside of the 'with' block. So, there's really no need to hold the @initrd_template file open longer than necessary (not that it would be a big deal anyway). [1] https://docs.python.org/3.7/reference/executionmodel.html Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 88eb248..dcc54ae 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -610,11 +610,11 @@ class Application: facts[option] ) =20 - tempdir =3D tempfile.mkdtemp() - initrd_inject =3D os.path.join(tempdir, install_config) + tempdir =3D tempfile.mkdtemp() + initrd_inject =3D os.path.join(tempdir, install_config) =20 - with open(initrd_inject, "w") as inject: - inject.write(content) + with open(initrd_inject, "w") as inject: + inject.write(content) =20 # preseed files must use a well-known name to be picked up by # d-i; for kickstart files, we can use whatever name we please --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294562; cv=none; d=zohomail.com; s=zohoarc; b=Bqswv58aWzr1Yu0Y3b7fZduqGEsrg0ErGWHvav/8aLf1idvyALCDWYdJZRChIqNgtErKSWPxf+8CxKQU3NTWbHPm/2CJfKttMSVg/+jGTIDMGSaQiK4FQdM5iyEkZpfhmCsmWcrpA2RD7OKRNswmnZIOd7bET6usdoplz8/VRJg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294562; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LXz548DvIhsMgOOmgXRFirdSgjUizhBoQuZIkuQ8+Z4=; b=cuZz/VZvdW4Tdq+gApH4BHPp4O9eMy5Y3a8fTLUsqqmTGopDJi2QdG8Z6vhPSB4kRurivX/gFQMbAV9uiK9eoWcLrvVbXiOiZFp+p7HXxB+pOQnEojiPsKvDrOwD0Hii87n3dKrC6AUp26FWYAknvf/GrHSEpeJOp7ytSjo3UA4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1589294562602817.429420036293; Tue, 12 May 2020 07:42:42 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-96-hD6BiCj5Mye0S_6_9xQApg-1; Tue, 12 May 2020 10:42:37 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 54AA6835B40; Tue, 12 May 2020 14:42:32 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2E9701D4; Tue, 12 May 2020 14:42:32 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id F05874CAA0; Tue, 12 May 2020 14:42:31 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgUCX020162 for ; Tue, 12 May 2020 10:42:30 -0400 Received: by smtp.corp.redhat.com (Postfix) id 0352B5C1D2; Tue, 12 May 2020 14:42:30 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1E6185C1BB; Tue, 12 May 2020 14:42:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294561; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=LXz548DvIhsMgOOmgXRFirdSgjUizhBoQuZIkuQ8+Z4=; b=YzdrCscs63SLDUFIYmtY0EbmlZ/phmyp09ffuv4wJxnHcab/d/NI/mlet27y47O68CnWFy Z5FHSmEd7Z2fxerNbqV596S/Gk5cjACrI5WYwUBvqh4SMv+r84sXmnWazTuwoWt+Wyl67a 45QjO3+aiUqBM45k53OJq8dljjrwX9Y= X-MC-Unique: hD6BiCj5Mye0S_6_9xQApg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 03/13] lcitool: Prefer tempfile's native wrappers over low level primitives Date: Tue, 12 May 2020 16:42:09 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Rather than requiring shutil module to get rid of the temporary directory we're creating for virt-install, let's use the TemporaryDirectory method instead which returns a file-like object which can be used to clean up the standard Python way. Although the internal exit handlers will take care of closing the temporary directories (and thus removing their contents) automatically, let's be explicit anyway and use the 'finally' clause to clean these up on both success and failure. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index dcc54ae..56eb543 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -24,7 +24,6 @@ import json import os import platform import random -import shutil import string import subprocess import sys @@ -610,8 +609,8 @@ class Application: facts[option] ) =20 - tempdir =3D tempfile.mkdtemp() - initrd_inject =3D os.path.join(tempdir, install_config) + tempdir =3D tempfile.TemporaryDirectory(prefix=3D"lcitool") + initrd_inject =3D os.path.join(tempdir.name, install_config) =20 with open(initrd_inject, "w") as inject: inject.write(content) @@ -665,8 +664,8 @@ class Application: subprocess.check_call(cmd) except Exception as ex: raise Exception("Failed to install '{}': {}".format(host, = ex)) - - shutil.rmtree(tempdir, ignore_errors=3DTrue) + finally: + tempdir.cleanup() =20 def _action_update(self, args): self._execute_playbook("update", args.hosts, args.projects, --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294562; cv=none; d=zohomail.com; s=zohoarc; b=B7jpldcBvC8fyINu7u1LCE6fuq3N/jooJiShx9nEalrliZQo8Lice+RURD2j5YLESaXKhMYfyqGLrDnSPwaPfQrA0BSN5AS1cUEZwqTbh3SbGbU3A0Ey2KCU9fUiIc+J5iqGGt7ZfeZOsTo53cSf6UyLkvJYIPJmbMDYHgvH4qM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294562; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=br9EeEh1E2vhnawVvEOUy4RHsfdHw1u/a6vEyclqGwc=; b=Hx34rbezad5SFgemCRph8HMJFctxMj5AVfNm4B/QXlRiWNBHBLOjCWDj+vp0gKOg9YdLw9+MZmi8uEGvvvu5i3BPQZkEkUBfRg9eWC2G6xXZPOzSeAmNiWUcMYyatVwjKr25a3FWgHAQpX6eAkaxPkwqNETd1XrI9qISq+Ccx9k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294562611912.8966715396617; Tue, 12 May 2020 07:42:42 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-214-0THieQWMMjimDuIHqSt9gg-1; Tue, 12 May 2020 10:42:38 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 34A6B1B18BCC; Tue, 12 May 2020 14:42:33 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1136060BF1; Tue, 12 May 2020 14:42:33 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CD0B8180954D; Tue, 12 May 2020 14:42:32 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgVIf020172 for ; Tue, 12 May 2020 10:42:31 -0400 Received: by smtp.corp.redhat.com (Postfix) id 737615C1D3; Tue, 12 May 2020 14:42:31 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id 576895C1BB; Tue, 12 May 2020 14:42:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294561; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=br9EeEh1E2vhnawVvEOUy4RHsfdHw1u/a6vEyclqGwc=; b=faPnFrZmUtxmS+U8E372GR3toON6IRRP9M62oHiL3cVFAWcnwdCZTIx5T4cj/RQKV4tNTO KzjlnvN8gzkT04/ygdUqWr1phbAwhg9oHQZMcfMI9IqkXHfB2dToISy60iXduC74OQRl6H 7O+neHiuSmE9FuRsf39zO2Gpf8nX18M= X-MC-Unique: 0THieQWMMjimDuIHqSt9gg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 04/13] lcitool: Use a temporary JSON file to pass extra variables Date: Tue, 12 May 2020 16:42:10 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This patch is a pre-requisite config file consolidation. Currently we've got a number of files which serve as a configuration either to the lcitool itself or to the ansible playbooks (majority). Once we replace these with a single global lcitool config, we'd end up passing tokens (potentially some passwords) as ansible extra variables bare naked on the cmdline. In order to prevent this security flaw use temporary JSON file holding all these extra variables and pass it as follows: $ ansible-playbook --extra-vars @extra_vars.json playbook.yml Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 56eb543..5b44582 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -506,11 +506,14 @@ class Application: git_remote =3D "default" git_branch =3D "master" =20 + tempdir =3D tempfile.TemporaryDirectory(prefix=3D"lcitool") + ansible_cfg_path =3D os.path.join(base, "ansible.cfg") playbook_base =3D os.path.join(base, "playbooks", playbook) playbook_path =3D os.path.join(playbook_base, "main.yml") + extra_vars_path =3D os.path.join(tempdir.name, "extra_vars.json") =20 - extra_vars =3D json.dumps({ + extra_vars =3D { "base": base, "playbook_base": playbook_base, "root_password_file": root_pass_file, @@ -520,7 +523,10 @@ class Application: "git_branch": git_branch, "gitlab_url_file": gitlab_url_file, "gitlab_runner_token_file": gitlab_runner_token_file, - }) + } + + with open(extra_vars_path, "w") as fp: + json.dump(extra_vars, fp) =20 ansible_playbook =3D distutils.spawn.find_executable("ansible-play= book") if ansible_playbook is None: @@ -529,7 +535,7 @@ class Application: cmd =3D [ ansible_playbook, "--limit", ansible_hosts, - "--extra-vars", extra_vars, + "--extra-vars", "@" + extra_vars_path, ] =20 # Provide the vault password if available @@ -548,6 +554,8 @@ class Application: except Exception as ex: raise Exception( "Failed to run {} on '{}': {}".format(playbook, hosts, ex)) + finally: + tempdir.cleanup() =20 def _action_hosts(self, args): for host in self._inventory.expand_pattern("all"): --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294573; cv=none; d=zohomail.com; s=zohoarc; b=e12tFQSFa0u17hNZESJYhO1ifkcs216pdKzovwcDoW/xgzlbj693hNIopZKj4h0c1p39tjaw/jbOVplU2K+Q5IAsyPyyM9dzGfwb9fM5cYDzaaMrBqkwNvuAaAqm1mEe6XpgIUZVxlpsOfdCkbI3OPXT4GX8kATWbZUlymSw3Uw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294573; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=O48o/a7arZ0hNxXULgQRx0wVgSp4DaL42SAdbrmIdvU=; b=c6lNh8J2IVRdKaOCAIpNSgPxxPiKRnqBUKZ2KrV8govnrw2xy5A0bL858TQpW57dgIXDRFH5s7VZAUFuA/+gK+4GQbxEamXs/smLd9p3m4yuOmKmiripuXxbVjCB7DOzllAg+M94nMnTkz+5l+IB9cjFyNNwoOaP1LTksLpFaqk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294573692409.89735674376504; Tue, 12 May 2020 07:42:53 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-EUz_ZwLOPH-pNICJaAEMXw-1; Tue, 12 May 2020 10:42:48 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9D1E4107ACF4; Tue, 12 May 2020 14:42:41 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7E4CF7D918; Tue, 12 May 2020 14:42:41 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 461451806B0C; Tue, 12 May 2020 14:42:41 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgW0a020181 for ; Tue, 12 May 2020 10:42:32 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7DF345C1D3; Tue, 12 May 2020 14:42:32 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id C44855C1BB; Tue, 12 May 2020 14:42:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294572; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=O48o/a7arZ0hNxXULgQRx0wVgSp4DaL42SAdbrmIdvU=; b=UoDvdgahtY1cr+3eQ6cQ8X+yCYoSxzRw4qBHOqqMmqkgZLW6Qv2xKSBq0ZUZu27t4fTORu xAyO4tba9CD904gf1CrRNI1D0pvjLS2sptEgLsftgj6oV3Yy7BtTJL3n1PnohMsFmI8HXg dV6A37SoRzSzWuxMPuTSVwie7920NyU= X-MC-Unique: EUz_ZwLOPH-pNICJaAEMXw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 05/13] config: Introduce a new global config.yaml configuration file Date: Tue, 12 May 2020 16:42:11 +0200 Message-Id: <36a1b14a8320f0f591fc401f9e686f058aedad4f.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Rather than having the configuration options split across multiple files (root-password, flavor, gitlab-url, gitlab-runner-token, ...), let's consolidate these settings into a global config file. The YAML format has been chosen simply because it's the native data format in Ansible and thus plays very nicely when accessing variables within Ansible playbooks. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/config.yaml | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 guests/config.yaml diff --git a/guests/config.yaml b/guests/config.yaml new file mode 100644 index 0000000..05f1e8a --- /dev/null +++ b/guests/config.yaml @@ -0,0 +1,25 @@ +--- +# Configuration file for lcitool + +install: + + # Installation flavor determining the target environment for the VM: + # + # test - VMs suitable for local testing (the 'test' user has sudo privil= eges) + # jenkins - VMs ready to be plugged into a Jenkins environment + # gitlab - VMs ready to be plugged into a GitLab environment + flavor: test + + # Root password for the VM. This password will only be necessary for ser= ial + # console access in case something goes horribly wrong, for all other use + # cases, SSH key authentication will be used instead. (Mandatory) + root_password: + +# GitLab-related options (only apply to the 'gitlab' flavor) +gitlab: + + # GitLab connection URL + url: https://gitlab.com + + # GitLab runner registration token. (Mandatory) + runner_secret: --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294566; cv=none; d=zohomail.com; s=zohoarc; b=aht30X+V0pbZnLZBsExyZTnEPA7BLGAY/PJbNAEc/CaAvtP19TRHGN3bi9R+K6ECduYSmjxHudeEzSPZq91BNwIoouzQVzhj0M7fXSbZk4zJigZuhUnelAX2cVjv6ZuloOuJKfsl+V8rMA64RYUsrrviPgE7+MIVzSBvdcKfw9E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294566; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VL3i1JMbvKM0jPaB/foTPgQDobR2uTrSv1NSPEL2yF0=; b=ggTY+pArqR0oF20kFPqnbXJV+ROhy90xb/O+syB6unrwDPvgdoII++UBtWtz+l70bHPlNoUCez9kuxsOpm0lloYUYKPp1RFnyoctbnFre8SpNfvRgjr32+xEjG3sFR/RwFdhabqXhFQ30HtgpR3XyQSJpo7RysyKBOfWm4HFBkI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294566900280.1685910063643; Tue, 12 May 2020 07:42:46 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-228-Wd1ZnrHoPvS1yFH_7qouww-1; Tue, 12 May 2020 10:42:42 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DF5D1108BC53; Tue, 12 May 2020 14:42:36 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BC4F35C1C3; Tue, 12 May 2020 14:42:36 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8B5924CAA0; Tue, 12 May 2020 14:42:36 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgXhk020193 for ; Tue, 12 May 2020 10:42:33 -0400 Received: by smtp.corp.redhat.com (Postfix) id 8CD675C1D2; Tue, 12 May 2020 14:42:33 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id D2D175C1C3; Tue, 12 May 2020 14:42:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294565; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=VL3i1JMbvKM0jPaB/foTPgQDobR2uTrSv1NSPEL2yF0=; b=XYGGi/3FoJTJd32Qt/aOC4UmAgycFFrzVWZrm6wz1uvxE0hVRTFwSztg/1KooQYbnq+86H UvR0Bk0cG3s2SLqJDVW8lGJSUQtuhAxAAqziMAoF1NRFdTm7QHzYIxqZr3XcXsq79QRmr4 WJrBRC0U282AQCzqx8OFaJVPW5XE3Wk= X-MC-Unique: Wd1ZnrHoPvS1yFH_7qouww-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 06/13] lcitool: Introduce methods to load and validate the YAML config Date: Tue, 12 May 2020 16:42:12 +0200 Message-Id: <2ca9cd51582f2f83cf0f5232f48f696dd91d11ab.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This patch introduce a set of class Config helper methods in order to parse and validate the new global YAML config. Currently, only 'install' and 'gitlab' sections are recognized with the flavor setting defaulting to "test" (backwards compatibility) and gitlab runner registration url defaulting to "https://gitlab.com"; the rest of the options are currently mandatory. Signed-off-by: Erik Skultety --- guests/lcitool | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 5b44582..577e9d2 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -128,6 +128,30 @@ class Util: =20 class Config: =20 + def __init__(self): + + # Load the template config containing the defaults first, this must + # always succeed. + # NOTE: we should load this from /usr/share once we start packaging + # lcitool + base =3D Util.get_base() + with open(os.path.join(base, "config.yaml"), "r") as fp: + self.values =3D yaml.safe_load(fp) + + try: + with open(self._get_config_file("config.yaml"), "r") as fp: + user_config =3D yaml.safe_load(fp) + except Exception as e: + raise Exception("Missing or invalid config.yaml file: {}".form= at(e)) + + if user_config is None: + raise Exception("Missing or invalid config.yaml file") + + # Validate the user provided config and use it to override the def= ault + # settings + self._validate(user_config) + self._update(user_config) + @staticmethod def _get_config_file(name): try: @@ -149,6 +173,64 @@ class Config: =20 return os.path.join(config_dir, name) =20 + @staticmethod + def _remove_unknown_keys(_dict, known_keys): + keys =3D list(_dict.keys()) + + for k in keys: + if k not in known_keys: + del _dict[k] + + def _validate_section(self, config, section, mandatory_keys): + # remove keys we don't recognize + self._remove_unknown_keys(config[section], self.values[section].ke= ys()) + + # check that the mandatory keys are present and non-empty + for key in mandatory_keys: + if config.get(section).get(key, None) is None: + raise Exception(("Missing or empty value for mandatory key" + "'{}.{}'").format(section, key)) + + # check that all keys have values assigned and of the right type + for key in config[section].keys(): + + # mandatory keys were already checked, so this covers optional= keys + if config[section][key] is None: + raise Exception( + "Missing value for '{}.{}'".format(section, key) + ) + + if not isinstance(config[section][key], (str, int)): + raise Exception( + "Invalid type for key '{}.{}'".format(section, key) + ) + + def _validate(self, config): + # delete sections we don't recognize + self._remove_unknown_keys(config, self.values.keys()) + + if "install" not in config: + raise Exception("Missing mandatory section 'install'") + + self._validate_section(config, "install", ["root_password"]) + + # we only need this for the gitlab check below, if 'flavor' is mis= sing + # that's okay, we'll provide a default later + flavor =3D config["install"].get("flavor", None) + if flavor is not None and flavor not in ["test", "jenkins", "gitla= b"]: + raise Exception( + "Invalid value '{}' for 'install.flavor'".format(flavor) + ) + + if flavor =3D=3D "gitlab": + self._validate_section(config, "gitlab", ["runner_secret"]) + + def _update(self, values): + self.values["install"].update(values["install"]) + + if values.get("gitlab", None): + self.values["gitlab"].update(values["gitlab"]) + def get_flavor(self): flavor_file =3D self._get_config_file("flavor") =20 --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294594; cv=none; d=zohomail.com; s=zohoarc; b=a7Z+mWlt+iAG8ZcjAK4W2iP8VlEvbH+KjMytY4D7Z/ZdhgFSNR/ZKoppCE4oFsCamgEyCMumYEDElSOu1BRHfeIgmYmF12+nG+mMhnimMDXr4ckjlEsZ/HHcXS//+qssh9ohgRfTL8OcgANWpQZQ9Vlv/Vf2W4+HdKxXWeBSK9E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294594; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LU/NCEDzqcI06dsvJj6IO1rZjG8cVd1tmCD/l1Lnf4g=; b=ekzE8KAkWqTeqWS1yHP+ZK23yz7jI5sx5/DuyKvcLz8UVik8sYj2XNHRgi4+zaTGeX0TWg4Gamu6T2ozt8dxcfHlBcKlkqIwzqRvgZqtVn742KPPT9W1zT0ah4y5467OnRtA8L7R/oHOom6WyO97wHISXVMrXhLa2pvFPGZzZ6U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294594081529.0854207150516; Tue, 12 May 2020 07:43:14 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-149--l6i34ArNOq8qaSdJknScQ-1; Tue, 12 May 2020 10:42:48 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3FA2D107ACF9; Tue, 12 May 2020 14:42:39 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 20AB66C791; Tue, 12 May 2020 14:42:39 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E03051809561; Tue, 12 May 2020 14:42:38 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgYYV020198 for ; Tue, 12 May 2020 10:42:34 -0400 Received: by smtp.corp.redhat.com (Postfix) id 9BD2E5C1D2; Tue, 12 May 2020 14:42:34 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id E19105C1BB; Tue, 12 May 2020 14:42:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294592; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=LU/NCEDzqcI06dsvJj6IO1rZjG8cVd1tmCD/l1Lnf4g=; b=RV14iGEfVYmhrtC3Lf3/QbNyE6csnYuNaUXlM1bb7F1141+LPwxIsccNq9NrLV8323rBOb oXU97A/AO28LPvDtcP+gK5e00FQM7R2RFtTnYDjqMZ1eED6iuPQ8uOPLCweNpeedj2Z529 yL8eyKacPRx4zUQSfAn7Pm7qElK+6qw= X-MC-Unique: -l6i34ArNOq8qaSdJknScQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 07/13] lcitool: Update the config values with internal playbook settings Date: Tue, 12 May 2020 16:42:13 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" So, the idea is to pass our YAML config to the Ansible playbooks as extra vars. However, not all variables we need to pass on to Ansible are exposed in the config (and they shouldn't be). Update the config values dictionary with these variables before passing on to Ansible. Signed-off-by: Erik Skultety --- guests/lcitool | 10 ++---- guests/playbooks/build/main.yml | 2 +- guests/playbooks/update/main.yml | 6 ++-- guests/playbooks/update/tasks/gitlab.yml | 4 +-- guests/playbooks/update/tasks/kludges.yml | 2 +- guests/playbooks/update/tasks/users.yml | 42 +++++++++++------------ 6 files changed, 30 insertions(+), 36 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 577e9d2..3270fca 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -595,20 +595,16 @@ class Application: playbook_path =3D os.path.join(playbook_base, "main.yml") extra_vars_path =3D os.path.join(tempdir.name, "extra_vars.json") =20 - extra_vars =3D { + self._config.values.update({ "base": base, "playbook_base": playbook_base, - "root_password_file": root_pass_file, - "flavor": flavor, "selected_projects": selected_projects, "git_remote": git_remote, "git_branch": git_branch, - "gitlab_url_file": gitlab_url_file, - "gitlab_runner_token_file": gitlab_runner_token_file, - } + }) =20 with open(extra_vars_path, "w") as fp: - json.dump(extra_vars, fp) + json.dump(self._config.values, fp) =20 ansible_playbook =3D distutils.spawn.find_executable("ansible-play= book") if ansible_playbook is None: diff --git a/guests/playbooks/build/main.yml b/guests/playbooks/build/main.= yml index 8abda67..462764b 100644 --- a/guests/playbooks/build/main.yml +++ b/guests/playbooks/build/main.yml @@ -1,6 +1,6 @@ --- - hosts: all - remote_user: '{{ flavor }}' + remote_user: '{{ install.flavor }}' =20 vars_files: - '{{ playbook_base }}/jobs/defaults.yml' diff --git a/guests/playbooks/update/main.yml b/guests/playbooks/update/mai= n.yml index 371e53d..1b97027 100644 --- a/guests/playbooks/update/main.yml +++ b/guests/playbooks/update/main.yml @@ -45,7 +45,7 @@ vars: project: jenkins when: - - flavor =3D=3D "jenkins" + - install.flavor =3D=3D "jenkins" =20 # Configure environment. Needs to happen after installing packages - include: '{{ playbook_base }}/tasks/kludges.yml' @@ -57,9 +57,9 @@ # Configure the Jenkins agent - include: '{{ playbook_base }}/tasks/jenkins.yml' when: - - flavor =3D=3D 'jenkins' + - install.flavor =3D=3D 'jenkins' =20 # Install the Gitlab runner agent - include: '{{ playbook_base }}/tasks/gitlab.yml' when: - - flavor =3D=3D 'gitlab' + - install.flavor =3D=3D 'gitlab' diff --git a/guests/playbooks/update/tasks/gitlab.yml b/guests/playbooks/up= date/tasks/gitlab.yml index f07279c..07a376c 100644 --- a/guests/playbooks/update/tasks/gitlab.yml +++ b/guests/playbooks/update/tasks/gitlab.yml @@ -1,8 +1,6 @@ --- - name: Define gitlab-related facts set_fact: - gitlab_url: '{{ lookup("file", gitlab_url_file) }}' - gitlab_runner_secret: '{{ lookup("file", gitlab_runner_token_file) }}' gitlab_runner_download_url: https://gitlab-runner-downloads.s3.amazona= ws.com/latest/binaries/gitlab-runner-{{ ansible_system|lower }}-amd64 gitlab_runner_config_dir: '/etc/gitlab-runner' =20 @@ -14,7 +12,7 @@ force: yes =20 - name: Register the gitlab-runner agent - shell: 'gitlab-runner register --non-interactive --config "{{ gitlab_run= ner_config_dir }}/config.toml" --registration-token "{{ gitlab_runner_secre= t }}" --url "{{ gitlab_url }}" --executor shell --tag-list "{{ os.name|lowe= r }}-{{ os.version }}"' + shell: 'gitlab-runner register --non-interactive --config "{{ gitlab_run= ner_config_dir }}/config.toml" --registration-token "{{ gitlab.runner_secre= t }}" --url "{{ gitlab.url }}" --executor shell --tag-list "{{ os.name|lowe= r }}-{{ os.version }}"' args: creates: '{{ gitlab_runner_config_dir }}/config.toml' =20 diff --git a/guests/playbooks/update/tasks/kludges.yml b/guests/playbooks/u= pdate/tasks/kludges.yml index 96fe1a5..33c6532 100644 --- a/guests/playbooks/update/tasks/kludges.yml +++ b/guests/playbooks/update/tasks/kludges.yml @@ -12,7 +12,7 @@ group: wheel when: - os.name =3D=3D 'FreeBSD' - - flavor =3D=3D "jenkins" + - install.flavor =3D=3D "jenkins" =20 # FreeBSD compiles bash without defining SSH_SOURCE_BASHRC, which means # it won't try to detect when it's spawned by ssh and source ~/.bashrc diff --git a/guests/playbooks/update/tasks/users.yml b/guests/playbooks/upd= ate/tasks/users.yml index 5c6ce8f..bc3cc11 100644 --- a/guests/playbooks/update/tasks/users.yml +++ b/guests/playbooks/update/tasks/users.yml @@ -2,7 +2,7 @@ - name: 'root: Set password' user: name: root - password: '{{ lookup("file", root_password_file)|password_hash("sha512= ") }}' + password: '{{ install.root_password|password_hash("sha512") }}' =20 - name: 'root: Configure ssh access' authorized_key: @@ -17,54 +17,54 @@ line: 'PermitRootLogin without-password' state: present =20 -- name: '{{ flavor }}: Create group' +- name: '{{ install.flavor }}: Create group' group: - name: '{{ flavor }}' + name: '{{ install.flavor }}' state: present =20 -- name: '{{ flavor }}: Create user account' +- name: '{{ install.flavor }}: Create user account' user: - name: '{{ flavor }}' - group: '{{ flavor }}' - comment: '{{ flavor }}' + name: '{{ install.flavor }}' + group: '{{ install.flavor }}' + comment: '{{ install.flavor }}' password: '*' shell: '{{ paths.bash }}' =20 -- name: '{{ flavor }}: Set password' +- name: '{{ install.flavor }}: Set password' user: - name: '{{ flavor }}' + name: '{{ install.flavor }}' password: '{{ "test"|password_hash("sha512") }}' when: - - flavor =3D=3D 'test' + - install.flavor =3D=3D 'test' =20 -- name: '{{ flavor }}: Configure ssh access' +- name: '{{ install.flavor }}: Configure ssh access' authorized_key: - user: '{{ flavor }}' + user: '{{ install.flavor }}' key: '{{ lookup("file", lookup("env", "HOME") + "/.ssh/id_rsa.pub") }}' state: present =20 -- name: '{{ flavor }}: Grant passwordless sudo access' +- name: '{{ install.flavor }}: Grant passwordless sudo access' lineinfile: path: '{{ paths.sudoers }}' - line: '{{ flavor }} ALL=3D(ALL) NOPASSWD: ALL' + line: '{{ install.flavor }} ALL=3D(ALL) NOPASSWD: ALL' state: present validate: 'visudo -cf %s' when: - - flavor =3D=3D 'test' + - install.flavor =3D=3D 'test' =20 -- name: '{{ flavor }}: Create shell profile' +- name: '{{ install.flavor }}: Create shell profile' template: src: '{{ playbook_base }}/templates/{{ item }}.j2' - dest: /home/{{ flavor }}/.{{ item }} - owner: '{{ flavor }}' - group: '{{ flavor }}' + dest: /home/{{ install.flavor }}/.{{ item }} + owner: '{{ install.flavor }}' + group: '{{ install.flavor }}' with_items: - bash_profile - bashrc =20 -- name: '{{ flavor }}: Remove unnecessary home skeleton files' +- name: '{{ install.flavor }}: Remove unnecessary home skeleton files' file: - path: /home/{{ flavor }}/.{{ item }} + path: /home/{{ install.flavor }}/.{{ item }} state: absent with_items: - profile --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294650; cv=none; d=zohomail.com; s=zohoarc; b=cmekcFPUQ9kpeDyLNC8QFOHUe1+y3B9uPy+t+QAMlexUH8I9SloX5PL2LjD5Fv7UG10LH/NzpBstYUvmAW1KEL9NckObbOeGYbx3vIEkOF0fEmR1nS4PDKZtO2Y/zbRQmyNEurcDqVSuqeJcLuGdpLL5Yvpq7TQZvnTNJNx2sl8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294650; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=iG4gYdILgWVzKoa/E5m6LyEfKjHU2xIK/hCzEjAG+VA=; b=avIheXOu6Kybhlv3EHi0gTNKJ7LepIEMW609YFFGbdncBAHhbEdb821DkIb7SIKr0TUJoItnZMZVXvixJ/18dCom/ZG/CMHWjkcDmCHUlgRCwg77R3jFqIsaBtHlCODThezzR9bUhq4k5PTprcfiWU8RSpMMUiYtJ0fcNKWA6kE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 15892946500351017.4369137637889; Tue, 12 May 2020 07:44:10 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-30-lFAD4xWfPnapK7KpyaEqXQ-1; Tue, 12 May 2020 10:42:48 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D6B6F83DC0F; Tue, 12 May 2020 14:42:42 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B30AB101D480; Tue, 12 May 2020 14:42:42 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 840064E585; Tue, 12 May 2020 14:42:42 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgco2020221 for ; Tue, 12 May 2020 10:42:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id 9A2E05C1D6; Tue, 12 May 2020 14:42:38 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id F067F5C1BB; Tue, 12 May 2020 14:42:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294648; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=iG4gYdILgWVzKoa/E5m6LyEfKjHU2xIK/hCzEjAG+VA=; b=P3Xk4ox2uJ92y8Uw4HzpHWPXiGz24+BdMMvVKVNJjh4Ok12Smyb8TNL2zMfgOrdcpp9n8f NjUtpsFTdsCRs8tSTFEc/GD1idx6h68LkIyxX8uRAIR2x7mqO1NkG7nvr9MZPZLzipFZXe kLLTSgQro1bPZY4SHzGLrx01ShzT0CY= X-MC-Unique: lFAD4xWfPnapK7KpyaEqXQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 08/13] lcitool: Drop the get_flavor() method Date: Tue, 12 May 2020 16:42:14 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can now access this value directly in the config dictionary. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 36 ++++-------------------------------- 1 file changed, 4 insertions(+), 32 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 3270fca..fa60218 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -231,37 +231,12 @@ class Config: if values.get("gitlab", None): self.values["gitlab"].update(values["gitlab"]) =20 - def get_flavor(self): - flavor_file =3D self._get_config_file("flavor") - - try: - with open(flavor_file, "r") as infile: - flavor =3D infile.readline().strip() - except Exception: - # If the flavor has not been configured, we choose the default - # and store it on disk to ensure consistent behavior from now = on - flavor =3D "test" - try: - with open(flavor_file, "w") as infile: - infile.write("{}\n".format(flavor)) - except Exception as ex: - raise Exception( - "Can't write flavor file ({}): {}".format( - flavor_file, ex - ) - ) - - if flavor not in ["test", "jenkins", "gitlab"]: - raise Exception("Invalid flavor '{}'".format(flavor)) - - return flavor - def get_vault_password_file(self): vault_pass_file =3D None =20 # The vault password is only needed for the jenkins flavor, but in # that case we want to make sure there's *something* in there - if self.get_flavor() =3D=3D "jenkins": + if self.values["install"]["flavor"] =3D=3D "jenkins": vault_pass_file =3D self._get_config_file("vault-password") =20 try: @@ -294,7 +269,7 @@ class Config: return root_pass_file =20 def get_gitlab_runner_token_file(self): - if self.get_flavor() !=3D "gitlab": + if self.values["install"]["flavor"] !=3D "gitlab": return None =20 gitlab_runner_token_file =3D self._get_config_file("gitlab-runner-= token") @@ -313,7 +288,7 @@ class Config: return gitlab_runner_token_file =20 def get_gitlab_url_file(self): - if self.get_flavor() !=3D "gitlab": + if self.values["install"]["flavor"] !=3D "gitlab": return None =20 gitlab_url_file =3D self._get_config_file("gitlab-url") @@ -567,7 +542,6 @@ class Application: def _execute_playbook(self, playbook, hosts, projects, git_revision): base =3D Util.get_base() =20 - flavor =3D self._config.get_flavor() vault_pass_file =3D self._config.get_vault_password_file() root_pass_file =3D self._config.get_root_password_file() gitlab_url_file =3D self._config.get_gitlab_url_file() @@ -646,8 +620,6 @@ class Application: def _action_install(self, args): base =3D Util.get_base() =20 - flavor =3D self._config.get_flavor() - for host in self._inventory.expand_pattern(args.hosts): facts =3D self._inventory.get_facts(host) =20 @@ -743,7 +715,7 @@ class Application: cmd.append("--noautoconsole") =20 # Only configure autostart for the guest for the jenkins flavor - if flavor =3D=3D "jenkins": + if self._config.values["install"]["flavor"] =3D=3D "jenkins": cmd +=3D ["--autostart"] =20 try: --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294581; cv=none; d=zohomail.com; s=zohoarc; b=TyAHy6+s/nF6e4Ia8bJMxtdP1kxGs4WVLhonlJjCdrTJlSiMD8gROg578oPefrH1wVorU3CqXvEdJ2Wfr6Ll4G7eJOwzCPGSBVTV9J9HROeVVynvcVnExgkJBenQdF9E4TfjF6hXmhoEfjrIbJRtcw277HxLfv+sIuZ0iGFWY5A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294581; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=uj9YSbsR/+Jh2pHe2Koy9OviPbex9sfPvdA45fa2hUs=; b=dx6sBSLerduABcg4v2VeWMtuBhUiRtYART7Pdd+ytM1mnnW8MkXPaHhuO9O2iOofX4w0FMzbdW5qNT0alI8cdZqk3f92snoI/qXM8BojlTvE6sGQ9agNn+ZRTNu0weZsNL3o0nr7gDOKbzJkpSGNbygbRU9eFMXrxng5q9K78Mg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1589294581706917.6692354773371; Tue, 12 May 2020 07:43:01 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-436-iS5IniwbNZOK1iR0Ts8UZg-1; Tue, 12 May 2020 10:42:57 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A59E187308F; Tue, 12 May 2020 14:42:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 841975DA2A; Tue, 12 May 2020 14:42:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 52B7F4E588; Tue, 12 May 2020 14:42:52 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEgoRM020252 for ; Tue, 12 May 2020 10:42:50 -0400 Received: by smtp.corp.redhat.com (Postfix) id D1FC55C661; Tue, 12 May 2020 14:42:50 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF5395C3E7; Tue, 12 May 2020 14:42:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294580; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=uj9YSbsR/+Jh2pHe2Koy9OviPbex9sfPvdA45fa2hUs=; b=GK75gJonvla/Mv4GnbIJ2WCWeOjYRbcKTMoZ/Q/GWQL7egBzrG/CKUj64DpNfc1P14l85S 1JwUdKqbr6ljPXj7FNKkdloCzr+vzTfE6JDbP4xmtHs8ISkt4kCZDJN7hqSn9MyV24DSdx X98k23x7r9Wdcz3yQetwksdXTiZXzPs= X-MC-Unique: iS5IniwbNZOK1iR0Ts8UZg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 09/13] lcitool: Drop the get_root_password_file() method Date: Tue, 12 May 2020 16:42:15 +0200 Message-Id: <5419b124f08fdd91ed4a0922b69adf9bba07a8d5.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can now access this value directly in the config dictionary. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 17 ----------------- 1 file changed, 17 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index fa60218..9dd4aea 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -252,22 +252,6 @@ class Config: =20 return vault_pass_file =20 - def get_root_password_file(self): - root_pass_file =3D self._get_config_file("root-password") - - try: - with open(root_pass_file, "r") as infile: - if not infile.readline().strip(): - raise ValueError - except Exception as ex: - raise Exception( - "Missing or invalid root password file ({}): {}".format( - root_pass_file, ex - ) - ) - - return root_pass_file - def get_gitlab_runner_token_file(self): if self.values["install"]["flavor"] !=3D "gitlab": return None @@ -543,7 +527,6 @@ class Application: base =3D Util.get_base() =20 vault_pass_file =3D self._config.get_vault_password_file() - root_pass_file =3D self._config.get_root_password_file() gitlab_url_file =3D self._config.get_gitlab_url_file() gitlab_runner_token_file =3D self._config.get_gitlab_runner_token_= file() =20 --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294592; cv=none; d=zohomail.com; s=zohoarc; b=l5aZiS06Cx+k7r/Ftxk6Jmk1FdL7vL+GFvl5L5RBpZiKbc2NIm9SwddsMlTnybKpBvy33A1Bb8ey59FKh1FirxLgAZHcF9LwBWnQ1JIkrbAq5HmMNYsV6oW9eQxiOoYJqPGKJtRIh413ZOuRkB7VO5ESSIXXNctEluJjeSjgCWY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294592; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=g9Z6l3IEaz8dqKNl3/Tk/HjK8nRJkWN6Ih+QC5IWbKA=; b=aUfxKeotCHG3dBumLfRcVzUkYo+dPxSnv9pQonmK4hnzyWeNyRVeBZMjHMz9rsZaSvbg2+xUbaTLI6HTyvS0OXlIyoLkze0If7Qs76G/OTiOwLNUhIEgBgNHzZhlPBBKT2h0Jv7Q5IIOFPIaxSYJowvOolqdUR1M8/NSz+xd5qQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294592297519.807016714764; Tue, 12 May 2020 07:43:12 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-348-a-w10f8pPk-HiXnO6sVpEw-1; Tue, 12 May 2020 10:43:08 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B0C6EC1A9; Tue, 12 May 2020 14:43:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id ED7F37D932; Tue, 12 May 2020 14:43:02 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B981C1809557; Tue, 12 May 2020 14:43:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEh1KM020285 for ; Tue, 12 May 2020 10:43:01 -0400 Received: by smtp.corp.redhat.com (Postfix) id B73965FC19; Tue, 12 May 2020 14:43:01 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5739C5C6D9; Tue, 12 May 2020 14:42:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294591; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=g9Z6l3IEaz8dqKNl3/Tk/HjK8nRJkWN6Ih+QC5IWbKA=; b=eiS6Rgu+Akly2GsHrmTVbMPmQnEOzIRzbsro7TCurX6PIdpbXxuwwzDtcvIQCZI3oN/aF1 V4rZtxr5NNFD3y72BxWmIaYpw2usPu1CvEA7aZtkhZl6qlkQWvz93ZMLbDQ5QVBevw8r0e 21gGcI5rgAW8VDS8jtFG7FGOS7revZw= X-MC-Unique: a-w10f8pPk-HiXnO6sVpEw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 10/13] lcitool: Drop the gitlab-related getter methods Date: Tue, 12 May 2020 16:42:16 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can now access the values directly in the config dictionary. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 40 ---------------------------------------- 1 file changed, 40 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 9dd4aea..a93b56e 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -252,44 +252,6 @@ class Config: =20 return vault_pass_file =20 - def get_gitlab_runner_token_file(self): - if self.values["install"]["flavor"] !=3D "gitlab": - return None - - gitlab_runner_token_file =3D self._get_config_file("gitlab-runner-= token") - - try: - with open(gitlab_runner_token_file, "r") as infile: - if not infile.readline().strip(): - raise ValueError - except Exception as ex: - raise Exception( - "Missing or invalid GitLab runner token file ({}): {}".for= mat( - gitlab_runner_token_file, ex - ) - ) - - return gitlab_runner_token_file - - def get_gitlab_url_file(self): - if self.values["install"]["flavor"] !=3D "gitlab": - return None - - gitlab_url_file =3D self._get_config_file("gitlab-url") - - try: - with open(gitlab_url_file, "r") as infile: - if not infile.readline().strip(): - raise ValueError - except Exception as ex: - raise Exception( - "Missing or invalid GitLab url file ({}): {}".format( - gitlab_url_file, ex - ) - ) - - return gitlab_url_file - =20 class Inventory: =20 @@ -527,8 +489,6 @@ class Application: base =3D Util.get_base() =20 vault_pass_file =3D self._config.get_vault_password_file() - gitlab_url_file =3D self._config.get_gitlab_url_file() - gitlab_runner_token_file =3D self._config.get_gitlab_runner_token_= file() =20 ansible_hosts =3D ",".join(self._inventory.expand_pattern(hosts)) selected_projects =3D self._projects.expand_pattern(projects) --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294609; cv=none; d=zohomail.com; s=zohoarc; b=VAoE5yx1Yc/3J4plReExk22B+6jghuy1vDo9CV4IRwEZV6l+tai0yXcuo7uSQhHI+VOiBud3DKML324BQ3wzvSajjpg062K64W/iAmwh1DRToLannfrMdUFti6KYg0Dx2zpBa3lim2GfaKD0K78tUtegWcDjPGc+mN5htm/D0mM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294609; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pzxfig8Mscj7/KmKAq+Hwqf3ouzYKKw5qwDjjkF5log=; b=SYkxUeucrQ0u3xrt6LBvCMWBRDLkl8TK/55FxM7pUsN5kghFuSgQ6SWBNb8oAncf54NZtUuMM4EXQpTdXxcbJjYIB9B3/bt0YAOh5gVZ+ZhJPJwa/N/MQi637Xo7BYfCIbYYJDCbYmFuCeVpncCgnwneXur/HwH90z0GgPoVyBc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1589294609109150.93956366401721; Tue, 12 May 2020 07:43:29 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-47-pUkrtw3dO62Vxg1xKhawnQ-1; Tue, 12 May 2020 10:43:21 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 84C38835B41; Tue, 12 May 2020 14:43:15 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 61E4278B20; Tue, 12 May 2020 14:43:15 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2FA3A4E984; Tue, 12 May 2020 14:43:15 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEhEvP020306 for ; Tue, 12 May 2020 10:43:14 -0400 Received: by smtp.corp.redhat.com (Postfix) id 1CD795FC0A; Tue, 12 May 2020 14:43:14 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC49D5C1C3; Tue, 12 May 2020 14:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294605; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=pzxfig8Mscj7/KmKAq+Hwqf3ouzYKKw5qwDjjkF5log=; b=U65IH4zE0AGwoPAif3Dm3nk7bdA0UMaa0KOxOO8M4SfpbNm99nclwdHKcaioJCrCgUkwGj jFT+EjS0Ln9JnQGUhOqrFz1xYIGFweloVZS/JRvvz8oTErsBmaBd4l2qUE1KE5QSZFWITm ru+denBKirQD+XYSyAjsJalBDdyfFCk= X-MC-Unique: pUkrtw3dO62Vxg1xKhawnQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 11/13] config: Move the virt-install settings from install.yml to the config Date: Tue, 12 May 2020 16:42:17 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Looking into the future where we're able to generate cloudinit images, we'll need to configure some of the install options which is currently not possible without editing the install.yml group vars file within the repository. That is suboptimal, so let's move the install options to the global config under the 'install' section so that further tweaking is possible (but explicitly discouraged at the same time). Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/config.yaml | 17 +++++++++++++++++ guests/group_vars/all/install.yml | 11 ----------- guests/lcitool | 21 +++++++++++---------- 3 files changed, 28 insertions(+), 21 deletions(-) delete mode 100644 guests/group_vars/all/install.yml diff --git a/guests/config.yaml b/guests/config.yaml index 05f1e8a..0b0b79c 100644 --- a/guests/config.yaml +++ b/guests/config.yaml @@ -15,6 +15,23 @@ install: # cases, SSH key authentication will be used instead. (Mandatory) root_password: =20 + # Settings mapping to the virt-install options - see virt-install(1). + # It is strongly recommended that you keep the following at their default + # values to produce machines which conform to the upstream libvirt stand= ard, + # unless you have a reason to do otherwise. + # + # Sizes are expressed in GiB. + # + virt_type: kvm + arch: x86_64 + machine: pc + cpu_model: host-passthrough + vcpus: 2 + memory_size: 2 + disk_size: 15 + storage_pool: default + network: default + # GitLab-related options (only apply to the 'gitlab' flavor) gitlab: =20 diff --git a/guests/group_vars/all/install.yml b/guests/group_vars/all/inst= all.yml deleted file mode 100644 index 94b752f..0000000 --- a/guests/group_vars/all/install.yml +++ /dev/null @@ -1,11 +0,0 @@ ---- -# Sizes are in GiB -install_virt_type: kvm -install_arch: x86_64 -install_machine: pc -install_cpu_model: host-passthrough -install_vcpus: 2 -install_memory_size: 2 -install_disk_size: 15 -install_storage_pool: default -install_network: default diff --git a/guests/lcitool b/guests/lcitool index a93b56e..378c937 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -562,6 +562,7 @@ class Application: =20 def _action_install(self, args): base =3D Util.get_base() + config =3D self._config =20 for host in self._inventory.expand_pattern(args.hosts): facts =3D self._inventory.get_facts(host) @@ -569,16 +570,16 @@ class Application: # Both memory size and disk size are stored as GiB in the # inventory, but virt-install expects the disk size in GiB # and the memory size in *MiB*, so perform conversion here - memory_arg =3D str(int(facts["install_memory_size"]) * 1024) + memory_arg =3D str(config.values["install"]["memory_size"] * 1= 024) =20 - vcpus_arg =3D str(facts["install_vcpus"]) + vcpus_arg =3D str(config.values["install"]["vcpus"]) =20 disk_arg =3D "size=3D{},pool=3D{},bus=3Dvirtio".format( - facts["install_disk_size"], - facts["install_storage_pool"], + config.values["install"]["disk_size"], + config.values["install"]["storage_pool"], ) network_arg =3D "network=3D{},model=3Dvirtio".format( - facts["install_network"], + config.values["install"]["network"], ) =20 # Different operating systems require different configuration @@ -638,10 +639,10 @@ class Application: virt_install, "--name", host, "--location", facts["install_url"], - "--virt-type", facts["install_virt_type"], - "--arch", facts["install_arch"], - "--machine", facts["install_machine"], - "--cpu", facts["install_cpu_model"], + "--virt-type", config.values["install"]["virt_type"], + "--arch", config.values["install"]["arch"], + "--machine", config.values["install"]["machine"], + "--cpu", config.values["install"]["cpu_model"], "--vcpus", vcpus_arg, "--memory", memory_arg, "--disk", disk_arg, @@ -658,7 +659,7 @@ class Application: cmd.append("--noautoconsole") =20 # Only configure autostart for the guest for the jenkins flavor - if self._config.values["install"]["flavor"] =3D=3D "jenkins": + if config.values["install"]["flavor"] =3D=3D "jenkins": cmd +=3D ["--autostart"] =20 try: --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294615; cv=none; d=zohomail.com; s=zohoarc; b=QKlZ3nJsC0Q0hlQ5lq3O1jbFqfMCL8VteUKcI5ildiuWwhbgU7KuH59jfnPhLNWeCC7/2wTAAkODY6Yh84hQHtbvf9twCT+N8TLBiXjEBgPeRDZQrojKSRrUDxA2mSqRthFWj3yk9Q1+3K0MoZJhr346BowTUEScWOnIPeqoV+g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294615; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dy2nrzvHk10kwRGahfCJM/F/0Pg8GEanvguH1XQX028=; b=GqIqHXLBQXGfRp4wp9AEI/sPaHtCGX4jZzTDLTopNqB9lVqJ0ZuXUANlEN3RL2eXmxYkc88isOtvVKiYqbGR3G5xoX47IF6QcGGp7XAlfZ/KCNhQU/TXHi3hhIG4opkBE/EJJbAeyAgITEJ2niAywk6/YYJLsc0dtPpmmn0ZLqk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1589294615114110.78805760513421; Tue, 12 May 2020 07:43:35 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-282-WASYp1VYP_Wfje_IUW45Jg-1; Tue, 12 May 2020 10:43:31 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C8F92EC1A1; Tue, 12 May 2020 14:43:25 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AAE897529C; Tue, 12 May 2020 14:43:25 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 796441800431; Tue, 12 May 2020 14:43:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEhNlG020337 for ; Tue, 12 May 2020 10:43:23 -0400 Received: by smtp.corp.redhat.com (Postfix) id 996F95C6D9; Tue, 12 May 2020 14:43:23 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id BE4055C1C3; Tue, 12 May 2020 14:43:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294613; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=dy2nrzvHk10kwRGahfCJM/F/0Pg8GEanvguH1XQX028=; b=XsHyA/16zsvoWOrdXqa7OXviHWkg5ei5KSSyaHyydU+TmDkNmCJ83yPe+AvWw5UgN0hJ4R 3YN43/2VqDQv0xYtU30zkc39RKue1NWBeq0lTR5i96SP5c6Riey3O3joTAVyeJmKtE0r3f LzKo/1pfXm0p9icR9d7sO8+UK3sOxJY= X-MC-Unique: WASYp1VYP_Wfje_IUW45Jg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 12/13] guests: README: Document the existence and usage of config.yaml Date: Tue, 12 May 2020 16:42:18 +0200 Message-Id: <6d9d4ae96203f071c00a2da38909468e48760960.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Andrea Bolognani , Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/README.markdown | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/guests/README.markdown b/guests/README.markdown index 7dbefed..0e70f5f 100644 --- a/guests/README.markdown +++ b/guests/README.markdown @@ -64,13 +64,15 @@ branches out of those with Host setup ---------- =20 -Ansible and `virt-install` need to be available on the host. +`ansible` and `virt-install` need to be available on the host, the former = can +be either installed system-wide using your package manager or using `pip` +(see the provided requirements.txt file). The latter can only be installed= with +your package manager as `virt-install` is not distributed via PyPI. =20 -Before you can start bringing up guests, you'll have to store your -site-specific root password in the `~/.config/lcitool/root-password` file. -This password will only be necessary for serial console access in case -something goes horribly wrong; for day to day operations, SSH key -authentication will be used instead. +Before you can start bringing up guests, you need to create +~/.config/lcitool/config.yaml, ideally by copying the `config.yaml` templa= te, +and set at least the options marked as "(mandatory)" depending on the flav= or +(`test`, `jenkins`, `gitlab`) you wish to use with your machines. =20 Ansible expects to be able to connect to the guests by name: installing and enabling the [libvirt NSS plugin](https://wiki.libvirt.org/page/NSS_module) @@ -123,8 +125,8 @@ Jenkins CI use -------------- =20 You'll need to configure `lcitool` to use the `jenkins` flavor for -guests: to do so, just write `jenkins` in the `~/.config/lcitool/flavor` -file. +guests. To do so, simply set the `install.flavor` to `jenkins` in +`~/.config/lcitool/config.yaml`. =20 Once a guest has been prepared, you'll be able to log in as root either via SSH (your public key will have been authorized) or on the serial --=20 2.25.3 From nobody Sat Apr 20 08:45:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589294624; cv=none; d=zohomail.com; s=zohoarc; b=PVtqJarytSdmOA9kNfU4o5c2wyDjUN53lnL/db+WFfCFgf24GZqoWyKHCMb4+XS3/R/3K8YlKmbyi8Bvs03NBP6KepLUOa8/UZQKh8FsB2VOzbSgRdi76RbV/6ft/mK0g0G4xK39fGvlPqTGH+fU09/M2o5gDUQ0gKwlFhZvZhk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589294624; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nQwjpYBmMHHyI90FefMZMd8rJFgKaWletaVuuZYeI30=; b=eir4d/5gwRJbmqeD/xnqCZEuaWSBy5PyeVrjMzFydc/BtIJ9C4nPRGz9R/KRhM5mulCS4InJTcm5OjK+sVhRcnzkbERBeGbqXFi8+Ui5Jy41azvTNX4Y334+HiINR9S0l/wZegoWHH1pCIaOm2bttYvv48hEO22EOqwPLNKu4tE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1589294624255438.7541805271941; Tue, 12 May 2020 07:43:44 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-456-DTkKPGFAPkSA2JSp1g6Oog-1; Tue, 12 May 2020 10:43:40 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 423AD108BD0C; Tue, 12 May 2020 14:43:35 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0E1915D9DD; Tue, 12 May 2020 14:43:35 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D240B4E988; Tue, 12 May 2020 14:43:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 04CEhXIU020362 for ; Tue, 12 May 2020 10:43:33 -0400 Received: by smtp.corp.redhat.com (Postfix) id A0F9818216; Tue, 12 May 2020 14:43:33 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.194.158]) by smtp.corp.redhat.com (Postfix) with ESMTP id AADAD5C1C3; Tue, 12 May 2020 14:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589294623; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=nQwjpYBmMHHyI90FefMZMd8rJFgKaWletaVuuZYeI30=; b=Et8TbTV8AadnlJn7JIjM2TtH82rD3KiHjGWyuAJkq9usrLyihdj52MVMOHpooN6oU2LdZp nv2leeBh8E08fYKG184+avPx6sA2qo5BeuMJdsy3g6qO3EYxY67Nobz1TYP+WewtXgpTtL iow8JEiWcDWdA6Nyjq1cYG0e36NzYyY= X-MC-Unique: DTkKPGFAPkSA2JSp1g6Oog-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH v2 13/13] DO NOT MERGE: Demonstrate functionality with pytest unit tests Date: Tue, 12 May 2020 16:42:19 +0200 Message-Id: <30405584e222e71674b29ae5222ca5f57ebe87ac.1589294534.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This patch exercises the functionality added to the Config class by defining a couple of pytest unit tests. The pytest script imports lcitool as a local module (which is far from ideal), but in order to do that, lcitool has to adopt the .py extension otherwise python refuses to import it, but then again, this patch is just to showcase the functionality. --- guests/{lcitool =3D> lcitool.py} | 0 guests/requirements.txt | 2 + guests/test_config.py | 165 +++++++++++++++++++++++++++++++++ 3 files changed, 167 insertions(+) rename guests/{lcitool =3D> lcitool.py} (100%) create mode 100644 guests/test_config.py diff --git a/guests/lcitool b/guests/lcitool.py similarity index 100% rename from guests/lcitool rename to guests/lcitool.py diff --git a/guests/requirements.txt b/guests/requirements.txt index 900cfd6..c2b8f2c 100644 --- a/guests/requirements.txt +++ b/guests/requirements.txt @@ -1,2 +1,4 @@ ansible jenkins-job-builder +pytest +mock diff --git a/guests/test_config.py b/guests/test_config.py new file mode 100644 index 0000000..09343cc --- /dev/null +++ b/guests/test_config.py @@ -0,0 +1,165 @@ +import os +import pytest +import yaml +from mock import patch +import lcitool + +empty_config =3D "" + +missing_mandatory_key =3D \ + """ + install: + flavor: test + """ + +empty_mandatory_key =3D \ + """ + install: + flavor: test + root_password: + """ + +empty_optional_key =3D \ + """ + install: + flavor: + root_password: foo + """ + +extra_key =3D \ + """ + install: + root_password: foo + bar: baz + """ + +empty_flavor =3D \ + """ + install: + root_password: foo + """ + +invalid_flavor =3D \ + """ + install: + flavor: bar + root_password: foo + """ + +invalid_value_type =3D \ + """ + install: + flavor: bar + root_password: + - let + - me + - in + """ + + +def Config_init_mock(self): + # load only the default config template here + base =3D lcitool.Util.get_base() + with open(os.path.join(base, "config.yaml"), "r") as fp: + self.values =3D yaml.safe_load(fp) + + +@pytest.fixture +def config(): + with patch.object(config, "__init__", Config_init_mock): + return lcitool.Config() + + +@pytest.mark.parametrize("yamlstr,_pass", [ + (empty_config, False), + (missing_mandatory_key, False), + (empty_mandatory_key, False), + (empty_optional_key, False), + (invalid_flavor, False), + (invalid_value_type, False), + (extra_key, True), + (empty_flavor, True), +]) +def test_config_validate(config, yamlstr, _pass): + user_config =3D yaml.safe_load(yamlstr) + + if _pass: + config._validate(user_config) + else: + with pytest.raises(Exception): + config._validate(user_config) + + +yaml_in1 =3D \ + """ + install: + root_password: foo + vcpus: 4 + """ + +dict_out1 =3D { + "install": { + "flavor": "test", + "root_password": "foo", + "virt_type": "kvm", + "arch": "x86_64", + "machine": "pc", + "cpu_model": "host-passthrough", + "vcpus": 4, + "memory_size": 2, + "disk_size": 15, + "storage_pool": "default", + "network": "default"}, + "gitlab": { + "url": "https://gitlab.com", + "runner_secret": None} +} + +yaml_in2 =3D \ + """ + install: + flavor: gitlab + root_password: foo + virt_type: qemu + arch: aarch64 + machine: q35 + cpu_model: host-passthrough + vcpus: 4 + memory_size: 4 + disk_size: 8 + storage_pool: foo + network: bar + gitlab: + url: https://example.com + runner_secret: foobar + """ + +dict_out2 =3D { + "install": { + "flavor": "gitlab", + "root_password": "foo", + "virt_type": "qemu", + "arch": "aarch64", + "machine": "q35", + "cpu_model": "host-passthrough", + "vcpus": 4, + "memory_size": 4, + "disk_size": 8, + "storage_pool": "foo", + "network": "bar"}, + "gitlab": { + "url": "https://example.com", + "runner_secret": "foobar"} +} + + +@pytest.mark.parametrize("yaml_in, dict_out", [ + (yaml_in1, dict_out1), + (yaml_in2, dict_out2) +]) +def test_compare_config_contents(config, yaml_in, dict_out): + parsed =3D yaml.safe_load(yaml_in) + + # fill in the default values + config._update(parsed) + assert config.values =3D=3D dict_out --=20 2.25.3