From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562141; cv=none; d=zohomail.com; s=zohoarc; b=bn+PFYuuUp7eyU8yDf08KNFai1VZXevySRP0JRhGlE0xU7SLpjuIwmwVyVJ6zLFXjA83s4jKBUS6BHIMSU34uuzEdxNxJrt/ctoY8QOnI0xhR38muk3xB8gHeh/XlkY3BvPufrnP99Cx1/K3xRSM2+r1BfLbzIZUnH+K1VpPWUQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562141; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SFISxEy4z3sTC/CSgF73YyjImvY4HifRhnyP9664JM0=; b=ZOioLG3jBCXKA529D55B4YIaQPDhAWXiXBFAB4Hn8jueg/t9bYq9vG9Nnc8CaPhkptYUROeyekvmgIUr7BN5X1NDySGTDeSZr51BZbqXCToe3IJS5nowI1QqRDutiHqlOCNjUd/o6VCPep+IpYvIIZxPOxwpQwVOmz746uXug9k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1587562141333253.43317451390828; Wed, 22 Apr 2020 06:29:01 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-101-jgTnZCExNQq_EswD2nJ2ew-1; Wed, 22 Apr 2020 09:28:57 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CF12A1137845; Wed, 22 Apr 2020 13:28:51 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A5A215C1B2; Wed, 22 Apr 2020 13:28:51 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5B9E9941BC; Wed, 22 Apr 2020 13:28:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSbXs005786 for ; Wed, 22 Apr 2020 09:28:37 -0400 Received: by smtp.corp.redhat.com (Postfix) id 87C83608EE; Wed, 22 Apr 2020 13:28:37 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF9B86084A; Wed, 22 Apr 2020 13:28:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562140; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=SFISxEy4z3sTC/CSgF73YyjImvY4HifRhnyP9664JM0=; b=IyFGBeq5Kj/S/GJrvAp1lyRdx4rFSn5KRHAD8dXxtMVYsCI58lIMBguxFUrq0U1brEljZb 2zA6p6RKnXRafnUv9cBA244jhizFh9oXD+WEE49ILN4zeB7kprdtnyd7s9MUdhiHXAsnYF 6Rp46S7a8QHKdv1QLP1MWK5bIqtRtKo= X-MC-Unique: jgTnZCExNQq_EswD2nJ2ew-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 01/13] requirements: Introduce a requirements.txt file Date: Wed, 22 Apr 2020 15:28:19 +0200 Message-Id: <3551654ad6dfa375e474035e76c3b1f3cd6af9d5.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Right now, lcitool's users have 2 options to satisfy the tool's dependencies: install them through the system package manager system-wide or install through pip manually. Since pip gives us the option to automate this process a tiny bit, let's ship a requirements file. This targets users who are used to work with Python virtual environments and install whatever it is necessary inside. Signed-off-by: Erik Skultety --- requirements.txt | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 requirements.txt diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..dc416fa --- /dev/null +++ b/requirements.txt @@ -0,0 +1,3 @@ +ansible +jenkins-job-builder +toml --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562149; cv=none; d=zohomail.com; s=zohoarc; b=Nnhank3bRPciqoP+POEbFgGVCYi73Ugd5xsNd03PW72Hg53TniqtDGSM+JrB6+oAqs0JtmTlimpb28lhLaV8B9+9A1V0k8+2hBUefFIzw/Jy4FvS/MIMT9/DC1Nh6qZGXurMWAUwLK6qhlfi81cEs+xIbD4PIYapov4CmCLt0r8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562149; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9xRl7Vyg2AGpQ9CvWslKZeL3+mjFB1OsNfgl2IGG14w=; b=lJSZjhDlISZXGx83X9t1XnUwSfQr8IUpBqbGzKX6G9/lmHxiQXBnhwKHhHMn5FWnRD6cKdDjdzKDX2nnJsPKYzUh9WfHeLZdlwAvTC0qe6U679S5lgc2QiuFvAZU6mGjejL+BK+q8NVxNHyZovN1BvVy0fqx+PjC8RONeXAb0lc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1587562149930674.3244325315127; Wed, 22 Apr 2020 06:29:09 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-441-PCibP4DnM_2BVg9OFSCb0w-1; Wed, 22 Apr 2020 09:29:06 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E93E41922971; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BB39F5C57A; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6994C1809557; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDScoP005796 for ; Wed, 22 Apr 2020 09:28:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id A7F586084D; Wed, 22 Apr 2020 13:28:38 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id F02AA6084A; Wed, 22 Apr 2020 13:28:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562148; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9xRl7Vyg2AGpQ9CvWslKZeL3+mjFB1OsNfgl2IGG14w=; b=DZPaU8t/CJcd6Sv/uves7K/S9xmUr6bi730FLjsx6tY0A0Gt4PUFuxIEJzPxxbQe6ttq0K gL1J6ybnNMUQszdSMmE+7oBUQKa82bh6c9uh/cJIdFnPOBpxSbPkX3Rz/38RWO/xA9n2Q7 QOO9xgxEzaKrNOXY20xvl2Ja6asYvwM= X-MC-Unique: PCibP4DnM_2BVg9OFSCb0w-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 02/13] lcitool: Decrease the indent when creating a tempdir for initrd injection Date: Wed, 22 Apr 2020 15:28:20 +0200 Message-Id: <042d176655553295eacbd01dc00da17df8c675a3.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" The 'with' statement doesn't define a new code block [1], thus no local namespace is created. Therefore, we can still access the @content variable outside of the 'with' block. So, there's really no need to hold the @initrd_template file open longer than necessary (not that it would be a big deal anyway). [1] https://docs.python.org/3.7/reference/executionmodel.html Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index dfb1ebc..9a9326d 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -608,11 +608,11 @@ class Application: facts[option] ) =20 - tempdir =3D tempfile.mkdtemp() - initrd_inject =3D os.path.join(tempdir, install_config) + tempdir =3D tempfile.mkdtemp() + initrd_inject =3D os.path.join(tempdir, install_config) =20 - with open(initrd_inject, "w") as inject: - inject.write(content) + with open(initrd_inject, "w") as inject: + inject.write(content) =20 # preseed files must use a well-known name to be picked up by # d-i; for kickstart files, we can use whatever name we please --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562135; cv=none; d=zohomail.com; s=zohoarc; b=cAv7+QP/cnD9RCGCkU8VMx9Mp4iypXyGw8h1Od8GzXHlDHl4+uVy/Kf1au2cZpSoY96UrkNGWNmNMFpz+KNH4CGpjsCoMBYxKh0RUpQ4t+9AWUYwRtte4RNcjalyli949L984iEGjRirLxDLytrLPg6R/8eioO3mZX5N4W8bTJs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562135; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NKMcUrtxV/6IJ1xYDTX3bBnDWmlnjHqRGU8oWkPEFR4=; b=c4HUkCkth3mNGqdNRerrG2kPO5fKoKjbIF4nE0thld6tPrFxtFV2HCj5Qq/xoTRO/qP0g4rWZ+aUzAjaNJXcToNI32CpYPCRel2y4TkqlZoIhCnFIIZjMNVyPI4YlrgNkMf1L6cEUKaqhZ7GLxVvbgtGWLP90CAmU+Gb/zucYmM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1587562135289473.3631716849733; Wed, 22 Apr 2020 06:28:55 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-364-8vg-1uacM0m5g8ntkYgl8A-1; Wed, 22 Apr 2020 09:28:49 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4DE631922961; Wed, 22 Apr 2020 13:28:43 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 13D265C557; Wed, 22 Apr 2020 13:28:43 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8D5C31809543; Wed, 22 Apr 2020 13:28:42 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSd52005804 for ; Wed, 22 Apr 2020 09:28:39 -0400 Received: by smtp.corp.redhat.com (Postfix) id B738A6084B; Wed, 22 Apr 2020 13:28:39 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0BB066084A; Wed, 22 Apr 2020 13:28:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562133; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=NKMcUrtxV/6IJ1xYDTX3bBnDWmlnjHqRGU8oWkPEFR4=; b=dar0aXgWAXNyS2IvCibwDh5aJFAIrW68wf1stQRsQVtnKDJxQ9xFKcRCeTj9beuLr2tFum qoaH7yBJxG7e/JaxCzQAmHwAUXYZmq70Q08dWXy1/NAvflMozXoCdQDyemEEtCO7OYhE4M RoCwDr1Y5OS5kITD+qmASbih9WT8jTA= X-MC-Unique: 8vg-1uacM0m5g8ntkYgl8A-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 03/13] lcitool: Prefer tempfile's native wrappers over low level primitives Date: Wed, 22 Apr 2020 15:28:21 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Rather than requiring shutil module to get rid of the temporary directory we're creating for virt-install, let's use the TemporaryDirectory method instead which returns a file-like object which can be used to clean up the standard Python way. Although the internal exit handlers will take care of closing the temporary directories (and thus removing their contents) automatically, let's be explicit anyway and use the 'finally' clause to clean these up on both success and failure. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 9a9326d..51ee211 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -23,7 +23,6 @@ import json import os import platform import random -import shutil import string import subprocess import sys @@ -608,8 +607,8 @@ class Application: facts[option] ) =20 - tempdir =3D tempfile.mkdtemp() - initrd_inject =3D os.path.join(tempdir, install_config) + tempdir =3D tempfile.TemporaryDirectory() + initrd_inject =3D os.path.join(tempdir.name, install_config) =20 with open(initrd_inject, "w") as inject: inject.write(content) @@ -663,8 +662,8 @@ class Application: subprocess.check_call(cmd) except Exception as ex: raise Exception("Failed to install '{}': {}".format(host, = ex)) - - shutil.rmtree(tempdir, ignore_errors=3DTrue) + finally: + tempdir.cleanup() =20 def _action_update(self, args): self._execute_playbook("update", args.hosts, args.projects, --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562153; cv=none; d=zohomail.com; s=zohoarc; b=i+R8XWSzmYaJmo8sCllRL+tRZUhtqHinkZP15Y/2kyAC+uSAqqXeOOnylWO7Mb1oFxeKWhi8PU7n8g56o4CHgS+infbENkX6mzJJlg8PSwX+oDbWCsV0eU3BKS7LYJrKgbiXT0k5BZateQ1k16gprefY6NvaU7PiQ1NKJ0K0h1U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562153; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vS+1LTQ7gNBh84TYxbi3ZRnQWOCFOIj/U0DzsRSJ0Oc=; b=UPkCjpBm4zrZLbPGRpmKms0/xf34EkmFz9HIW/p02FpGDPAf3WNAKxb8RQRn/qxB/Gk9wzM2fVMDd3weO6dVHF/xqm5eXspqWEQf75yNCM6Dyoz+fFZdetMNXiREj1wV1Hwk+j8R2FK8dB5WN2KWCE7BMQIprQrd8eM/Xp3TmsE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1587562153297236.56064611557974; Wed, 22 Apr 2020 06:29:13 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-387-LaQYoqyiMDOY-XQ-83ID5g-1; Wed, 22 Apr 2020 09:29:08 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EACF0DB23; Wed, 22 Apr 2020 13:29:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B701D5D706; Wed, 22 Apr 2020 13:29:01 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 51A7B941BF; Wed, 22 Apr 2020 13:29:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSer4005814 for ; Wed, 22 Apr 2020 09:28:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id C78926084D; Wed, 22 Apr 2020 13:28:40 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 197EC6084A; Wed, 22 Apr 2020 13:28:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562152; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=vS+1LTQ7gNBh84TYxbi3ZRnQWOCFOIj/U0DzsRSJ0Oc=; b=FEnJ09c6SAohUgePQBSdDIJDBhm7xA5lrU1Nwy/DCqKY04fw3y+vOJ98Hn6ryXQSPTOay6 Tei4JV7ZsYOVESw0bYKayrVu7Yilg6NtLmBVFM+3OfX80X7DBkOps3Bf6KKXsSmzA6hgFC +h/p+Uhl64WTADj0H3nm6tK5k+vL8+A= X-MC-Unique: LaQYoqyiMDOY-XQ-83ID5g-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 04/13] lcitool: Use a temporary JSON file to pass extra variables Date: Wed, 22 Apr 2020 15:28:22 +0200 Message-Id: <864cbf035339b3dc056a4ca7b7a99af4969f37d8.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This patch is a pre-requisite config file consolidation. Currently we've got a number of files which serve as a configuration either to the lcitool itself or to the ansible playbooks (majority). Once we replace these with a single global lcitool config, we'd end up passing tokens (potentially some passwords) as ansible extra variables bare naked on the cmdline. In order to prevent this security flaw use temporary JSON file holding all these extra variables and pass it as follows: $ ansible-playbook --extra-vars @extra_vars.json playbook.yml Signed-off-by: Erik Skultety --- guests/lcitool | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 51ee211..138b5e2 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -504,21 +504,26 @@ class Application: git_remote =3D "default" git_branch =3D "master" =20 + tempdir =3D tempfile.TemporaryDirectory(prefix=3D'lcitool') + ansible_cfg_path =3D os.path.join(base, "ansible.cfg") playbook_base =3D os.path.join(base, "playbooks", playbook) playbook_path =3D os.path.join(playbook_base, "main.yml") + extra_vars_path =3D os.path.join(tempdir.name, 'extra_vars.json') =20 - extra_vars =3D json.dumps({ - "base": base, - "playbook_base": playbook_base, - "root_password_file": root_pass_file, - "flavor": flavor, - "selected_projects": selected_projects, - "git_remote": git_remote, - "git_branch": git_branch, - "gitlab_url_file": gitlab_url_file, - "gitlab_runner_token_file": gitlab_runner_token_file, - }) + with open(extra_vars_path, 'w') as fp: + extra_vars =3D { + "base": base, + "playbook_base": playbook_base, + "root_password_file": root_pass_file, + "flavor": flavor, + "selected_projects": selected_projects, + "git_remote": git_remote, + "git_branch": git_branch, + "gitlab_url_file": gitlab_url_file, + "gitlab_runner_token_file": gitlab_runner_token_file, + } + json.dump(extra_vars, fp) =20 ansible_playbook =3D distutils.spawn.find_executable("ansible-play= book") if ansible_playbook is None: @@ -527,7 +532,7 @@ class Application: cmd =3D [ ansible_playbook, "--limit", ansible_hosts, - "--extra-vars", extra_vars, + "--extra-vars", "@" + extra_vars_path, ] =20 # Provide the vault password if available @@ -546,6 +551,8 @@ class Application: except Exception as ex: raise Exception( "Failed to run {} on '{}': {}".format(playbook, hosts, ex)) + finally: + tempdir.cleanup() =20 def _action_hosts(self, args): for host in self._inventory.expand_pattern("all"): --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562204; cv=none; d=zohomail.com; s=zohoarc; b=EElOl2WkxYQ+NzDXwv1YdTZKqjxBZdlo/MREYvoTSiQWepOsMQCYJYeK6+cWjplUzPivo8Ghkql5RIbFtk7W0WF+bl2yaUG1BzqEX5Up1Wzsogi+/+8b2kjL043IjdrdyKTeVUSktOT3yTA2YxoXbEdcQi/AdFv6gmMtW3FgdsM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562204; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GGnzczISgtm5hx3POEbuGyaan7myVsWty//pctOMX+c=; b=QmSgXHLiuFrEyS+exHYElxeYxJ3Kl8aIsepDOxVutPYCrhuFM5tggL0zxfWDCI3CsHtmRBYfWfkbzeLlZ59iPaER4d0peVWO/tcCbBOhjYMEFrL0uUxyhrxBnst++wNNTemENB/l8z2uBaMkzZge0poSeLx96kGGFqSmtUEboSI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1587562204514921.1272780722461; Wed, 22 Apr 2020 06:30:04 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-328-OGAGDKxeM8WJU3Wy9W1H1Q-1; Wed, 22 Apr 2020 09:29:12 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 55D8B192296D; Wed, 22 Apr 2020 13:29:06 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2993B608E0; Wed, 22 Apr 2020 13:29:06 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CC127941C9; Wed, 22 Apr 2020 13:29:05 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSfgX005826 for ; Wed, 22 Apr 2020 09:28:41 -0400 Received: by smtp.corp.redhat.com (Postfix) id D342A608E0; Wed, 22 Apr 2020 13:28:41 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2777F6084D; Wed, 22 Apr 2020 13:28:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562203; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GGnzczISgtm5hx3POEbuGyaan7myVsWty//pctOMX+c=; b=XLY+Hz9oMszsbReRzsW/P2a0skmdrgJWN7ZSwzeRvFKysA4Yn8BUg/SlbjrFBhUwKsmmOe 4SMQtTayk9fm4rKj61ZpyebtpUaO0WVhe6tahuM6ijBtYjz+o62Q/ba3OR1y9FnGXvfCVr +wZcHlc6rg3Dv2rjOK31lCC7JHQe82E= X-MC-Unique: OGAGDKxeM8WJU3Wy9W1H1Q-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 05/13] config: Introduce a new global config.toml configuration file Date: Wed, 22 Apr 2020 15:28:23 +0200 Message-Id: <2a15f7380c271d06c641dfc765557f9972020360.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Rather than having the configuration options split across multiple files (root-password, flavor, gitlab-url, gitlab-runner-token, ...), let's consolidate these settings into a global config file. The TOML format has been chosen simply because its syntax is very similar to the conventional INI format (thus easily human readable), but unlike INI it has a formal specification, it's typed and it also has some nice features over the plain INI should we find ourselves in need of any of such extended features in the future, e.g. table nesting. Signed-off-by: Erik Skultety --- config.toml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 config.toml diff --git a/config.toml b/config.toml new file mode 100644 index 0000000..5bb3725 --- /dev/null +++ b/config.toml @@ -0,0 +1,26 @@ +# Configuration file for the lcitool -- https://gitlab.com/libvirt/libvirt= -ci/ +# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +[install] +# Installation flavor determining the target environment for the VM: +# +# test - VMs suitable for local testing, 'test' has passwordless sudo +# jenkins - VMs pluggable to a jenkins environment +# gitlab - VMs ready to be plugged to a GitLab environment +# +#flavor =3D "test" + +# Initial root password to be set by ansible on the appliance. This passwo= rd +# will only be necessary for serial console access in case something goes +# horribly wrong, for all other use cases, SSH key authentication will be = used +# instead. (mandatory) +#root_password =3D "" + +[gitlab] +# GitLab runner agent registration options, applies only if flavor =3D=3D = 'gitlab'. + +# GitLab server URL to register the runner. +#gitlab_url =3D "https://gitlab.com" + +# GitLab runner registration token. (mandatory) +#gitlab_runner_secret =3D "" --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562157; cv=none; d=zohomail.com; s=zohoarc; b=Gh29ZaVdUe2EvLAqBaKofEemRS0qbrdTxGIPTdWASLR477Lm+1ynE0YeMrARBnaYGpHCuBbFt0suBTAYLfTv8AzkRizDZfQphdrg1YmXaq2Bk/qh/1uVKKRxOurPo6GcL68RqpYmH5wVHTjCtlQCU/MtofScrGyqrL1Jz32lRCk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562157; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=g8etUzUF0xXiNsaQTnN2Mo/PQFRQvEb6wTmctww7eWY=; b=Bu5ffedHx2+nRxG9LRvhYMirfq17LQBagg3Nci5iY5+xEtSMVXoS+305N7v/uw4XPpbdXAe+KgivqlAzmxmuWYcwfej0Eypb2CuThwmCujSWYG3flRTW5QrIgbowZ+MlLR805OYgwv5H2/victiMYersqGcNAZnDrruiPiF0IVk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1587562157949923.047969624523; Wed, 22 Apr 2020 06:29:17 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-462-E4gqjZFpM26rnA-O1CpNWg-1; Wed, 22 Apr 2020 09:28:50 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 46F7C141F; Wed, 22 Apr 2020 13:28:45 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1922519756; Wed, 22 Apr 2020 13:28:45 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9788A1809554; Wed, 22 Apr 2020 13:28:44 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSgr0005837 for ; Wed, 22 Apr 2020 09:28:42 -0400 Received: by smtp.corp.redhat.com (Postfix) id E99D26084B; Wed, 22 Apr 2020 13:28:42 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33AA66084A; Wed, 22 Apr 2020 13:28:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562156; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=g8etUzUF0xXiNsaQTnN2Mo/PQFRQvEb6wTmctww7eWY=; b=cJ8kx3O+uCHmaXsbSPQucKuBWU7ClfRhGvkppcT1oiEyKh4yYO6YK1fvwsm8XiaWIiVxVp gjzvXrbRGiJhrV5zCem2GKJW7vwCE8i9NlcgeoEQCLXEicyCilRK3MxMJE6ACaasCddRze qJ86J7ChfE5z1zE9TxC2GL0gMp6O/MQ= X-MC-Unique: E4gqjZFpM26rnA-O1CpNWg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 06/13] lcitool: Introduce methods to load and validate the TOML config Date: Wed, 22 Apr 2020 15:28:24 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This patch introduce a set of class Config helper methods in order to parse and validate the new global TOML config. Currently, only 'install' and 'gitlab' sections are recognized with the flavor setting defaulting to "test" (backwards compatibility) and gitlab runner registration url defaulting to "https://gitlab.com"; the rest of the options are currently mandatory. Signed-off-by: Erik Skultety --- guests/lcitool | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 138b5e2..f08bd98 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -28,6 +28,7 @@ import subprocess import sys import tempfile import textwrap +import toml import yaml =20 # This is necessary to maintain Python 2.7 compatibility @@ -133,6 +134,21 @@ class Util: =20 class Config: =20 + def __init__(self): + path =3D self._get_config_file("config.toml") + + try: + self.dict =3D toml.load(path) + self._validate() + + except Exception as ex: + raise Exception( + "Missing or invalid config.toml file: {}".format(ex) + ) + + # fill in default options + self._fill_default_options(self.dict) + @staticmethod def _get_config_file(name): try: @@ -154,6 +170,42 @@ class Config: =20 return os.path.join(config_dir, name) =20 + @staticmethod + def _fill_default_options(cfg): + flavor =3D cfg.get("install").get("flavor", "test") + cfg["install"]["flavor"] =3D flavor + + if flavor =3D=3D "gitlab": + url =3D cfg.get("gitlab").get("gitlab_url", "https://gitlab.co= m") + cfg["gitlab"]["gitlab_url"] =3D url + + @staticmethod + def _validate_section(cfg, section, *args): + + if cfg.get(section, None) is None: + raise KeyError("Section '[{}]' not found".format(section)) + + for key in args: + if cfg.get(section).get(key, None) is None: + raise KeyError("Missing mandatory key '{}.{}'".format(sect= ion, + key)) + + def _validate(self): + + # verify the [install] section and its mandatory options + self._validate_section(self.dict, "install", "root_password") + + # we need to check flavor here, because later validations depend o= n it + flavor =3D self.dict.get("install").get("flavor", "test") + if flavor not in ["test", "jenkins", "gitlab"]: + raise ValueError( + "Invalid value '{}' for 'install.flavor'".format(flavor) + ) + + # verify the optional [gitlab] section and its mandatory options + if flavor =3D=3D "gitlab": + self._validate_section(self.dict, "gitlab", "gitlab_runner_sec= ret") + def get_flavor(self): flavor_file =3D self._get_config_file("flavor") =20 --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562148; cv=none; d=zohomail.com; s=zohoarc; b=GnvTTlTB5LsCfI0fT6nBx8lSOejT/Nhg16MhND1ZI6L9r12k17vKceziG/4qEmz1Qftwx8ImbybptQj6jZMX8/anEMlgvRj9dNGR/TEE3/mU9+OdEgj1IwgIQj5pxNqWu5w1+CoHzAI6Hm37P59cyZjZvEfjp5asdf+Fq3pkJFM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562148; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TiV9n3KYXyvTulAu/uyZcQRy+qtCiOlTuPIOrZpTqS8=; b=lzRnPD6GHbU4+z/QVqaA6fhjoi53jz6nlriLSgH/fd6aCk1R5sZvaWmuYWe7C5xVHQUaSX6c3w4W1CR6BmJib0qU6t+8BWcdGrteNx8Xp0C2YWbkT7CWfd+Nyzq0McBXQCpf8QjFwKpNe5HzWk6OI6ieWDakgQkqzzzDJij+H2o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1587562148106744.4165661792297; Wed, 22 Apr 2020 06:29:08 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-144-TjipM5RkMsyOS1Y0-P1jvg-1; Wed, 22 Apr 2020 09:29:03 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DD514100CCC4; Wed, 22 Apr 2020 13:28:53 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A8AF53A4; Wed, 22 Apr 2020 13:28:53 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 53055180B791; Wed, 22 Apr 2020 13:28:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDShp6005849 for ; Wed, 22 Apr 2020 09:28:43 -0400 Received: by smtp.corp.redhat.com (Postfix) id EAF0A6084D; Wed, 22 Apr 2020 13:28:43 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3FFD46084A; Wed, 22 Apr 2020 13:28:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562146; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=TiV9n3KYXyvTulAu/uyZcQRy+qtCiOlTuPIOrZpTqS8=; b=MvhfcpRgkwnU3yIJbIwTDBxVJXJtxdCkHonNnHgOQQHpxOrTntukpXmfKXTb0KpN36cPNw f1l0W7pKYZ+W8YihJZ5WorTnaSz0nm1mr/JOmokgv7dvRntQ43ohaxNhaL4cTgtWxFZrfJ ynWHkdiuXQTrrVcpdZti6/3fq8/BNds= X-MC-Unique: TjipM5RkMsyOS1Y0-P1jvg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 07/13] lcitool: Drop the get_flavor() method Date: Wed, 22 Apr 2020 15:28:25 +0200 Message-Id: <7aedaa90fd7cd7aed86769e2b5f62cb22e203039.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can now access this value directly in the config dictionary. Signed-off-by: Erik Skultety --- guests/lcitool | 32 +++----------------------------- 1 file changed, 3 insertions(+), 29 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index f08bd98..a385c15 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -206,37 +206,12 @@ class Config: if flavor =3D=3D "gitlab": self._validate_section(self.dict, "gitlab", "gitlab_runner_sec= ret") =20 - def get_flavor(self): - flavor_file =3D self._get_config_file("flavor") - - try: - with open(flavor_file, "r") as infile: - flavor =3D infile.readline().strip() - except Exception: - # If the flavor has not been configured, we choose the default - # and store it on disk to ensure consistent behavior from now = on - flavor =3D "test" - try: - with open(flavor_file, "w") as infile: - infile.write("{}\n".format(flavor)) - except Exception as ex: - raise Exception( - "Can't write flavor file ({}): {}".format( - flavor_file, ex - ) - ) - - if flavor not in ["test", "jenkins", "gitlab"]: - raise Exception("Invalid flavor '{}'".format(flavor)) - - return flavor - def get_vault_password_file(self): vault_pass_file =3D None =20 # The vault password is only needed for the jenkins flavor, but in # that case we want to make sure there's *something* in there - if self.get_flavor() =3D=3D "jenkins": + if self.dict["install"]["flavor"] =3D=3D "jenkins": vault_pass_file =3D self._get_config_file("vault-password") =20 try: @@ -535,7 +510,6 @@ class Application: def _execute_playbook(self, playbook, hosts, projects, git_revision): base =3D Util.get_base() =20 - flavor =3D self._config.get_flavor() vault_pass_file =3D self._config.get_vault_password_file() root_pass_file =3D self._config.get_root_password_file() gitlab_url_file =3D self._config.get_gitlab_url_file() @@ -568,7 +542,7 @@ class Application: "base": base, "playbook_base": playbook_base, "root_password_file": root_pass_file, - "flavor": flavor, + "flavor": self._config.dict["install"]["flavor"], "selected_projects": selected_projects, "git_remote": git_remote, "git_branch": git_branch, @@ -617,7 +591,7 @@ class Application: def _action_install(self, args): base =3D Util.get_base() =20 - flavor =3D self._config.get_flavor() + flavor =3D self._config.dict["install"]["flavor"] =20 for host in self._inventory.expand_pattern(args.hosts): facts =3D self._inventory.get_facts(host) --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562164; cv=none; d=zohomail.com; s=zohoarc; b=MPosKIe+ydyS51X3GMoSRNlnKoHiXmQ6hOKFJz2fvmtzygxaJV/RwvwZS/+5b1i63+itBuEoQLeqBxY1vpr/tpNfs3AOnhH5f+mskJOynUs9zfgfNfNBdAaAoIexO9yO06z2fzLpPJbuMlVanUNijMneIbufTbp0d5LmT9Nu3Ig= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562164; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=p+VnQXmMDsTvm8zCofhNthGfJnF0PDPXatuHBWz3Hxs=; b=KEyGfYllYkli8O5NhX7QlNHXLkYFeiuGP9rYjVEp9ZpR17ff7V5g7IWOVdzbx1WA949RC5DzW/asAdKcRUxMUN84H3goz1F1Xb+jucSIsDZv+rGiVmcmQJOsSvCe8c5nbLRU4EyRFL9KMuVzHMNqMzngRdocw0xzxNh8TITSRfA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1587562164668737.499861394357; Wed, 22 Apr 2020 06:29:24 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-117-H2CFAaAnM-OmMdD0rwY9Cw-1; Wed, 22 Apr 2020 09:29:18 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D3591137851; Wed, 22 Apr 2020 13:29:13 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 51290101E7F9; Wed, 22 Apr 2020 13:29:13 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id ED11E180CB27; Wed, 22 Apr 2020 13:29:12 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSjmh005855 for ; Wed, 22 Apr 2020 09:28:45 -0400 Received: by smtp.corp.redhat.com (Postfix) id 0991D608E0; Wed, 22 Apr 2020 13:28:45 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5023C6084D; Wed, 22 Apr 2020 13:28:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562163; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=p+VnQXmMDsTvm8zCofhNthGfJnF0PDPXatuHBWz3Hxs=; b=BeMx4ZJM0Jg3p5xr2fxP0TE1Q/gmXKr+Mp+u5gI2dkm/3M8Y7mLmP/XcIjaVKlVWUrAg9C +t6GSgcxZWBCZ32MKz05uOQVL7ToQYWNyXqHp+9FXbPJlw/aV4vD5DLH8ajifBkzhKKLYO W0ijkcrh9f6sPpnbs17JP47jtWQbQ5c= X-MC-Unique: H2CFAaAnM-OmMdD0rwY9Cw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 08/13] lcitool: Drop the get_root_password_file() method Date: Wed, 22 Apr 2020 15:28:26 +0200 Message-Id: <642fe0433ace782fa91c74683b513d46d42dd2db.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can now access this value directly in the config dictionary. Signed-off-by: Erik Skultety Reviewed-by: Andrea Bolognani --- guests/lcitool | 19 +------------------ guests/playbooks/update/tasks/users.yml | 2 +- 2 files changed, 2 insertions(+), 19 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index a385c15..6c905c2 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -227,22 +227,6 @@ class Config: =20 return vault_pass_file =20 - def get_root_password_file(self): - root_pass_file =3D self._get_config_file("root-password") - - try: - with open(root_pass_file, "r") as infile: - if not infile.readline().strip(): - raise ValueError - except Exception as ex: - raise Exception( - "Missing or invalid root password file ({}): {}".format( - root_pass_file, ex - ) - ) - - return root_pass_file - def get_gitlab_runner_token_file(self): if self.get_flavor() !=3D "gitlab": return None @@ -511,7 +495,6 @@ class Application: base =3D Util.get_base() =20 vault_pass_file =3D self._config.get_vault_password_file() - root_pass_file =3D self._config.get_root_password_file() gitlab_url_file =3D self._config.get_gitlab_url_file() gitlab_runner_token_file =3D self._config.get_gitlab_runner_token_= file() =20 @@ -541,7 +524,7 @@ class Application: extra_vars =3D { "base": base, "playbook_base": playbook_base, - "root_password_file": root_pass_file, + "root_password": self._config.dict["install"]["root_passwo= rd"], "flavor": self._config.dict["install"]["flavor"], "selected_projects": selected_projects, "git_remote": git_remote, diff --git a/guests/playbooks/update/tasks/users.yml b/guests/playbooks/upd= ate/tasks/users.yml index 28ee96a..8744290 100644 --- a/guests/playbooks/update/tasks/users.yml +++ b/guests/playbooks/update/tasks/users.yml @@ -2,7 +2,7 @@ - name: 'root: Set password' user: name: root - password: '{{ lookup("file", root_password_file)|password_hash("sha512= ") }}' + password: '{{ root_password|password_hash("sha512") }}' =20 - name: 'root: Configure ssh access' authorized_key: --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562176; cv=none; d=zohomail.com; s=zohoarc; b=P4q6UbKqu12VNQYf+PX7TsBx5Df8loLDUMHhgheWJIfXuVAer37qagvoB4aG5Hj21NwkuNqfJ3kVcWkQZfSwV/HwQ2fmYFVT41U9Eg++ZowQmNeZxc6tb6fo5CTOKiGiYpfdxgm5Um7khik0wKBUfhYO9lD2nJuHK1OUj04NtBg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562176; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4GSVH/yhxKpSFGXd453k9BuZvNL1DBOjvnKoTNdOABU=; b=lG7QdQByLU23tHGl1ReXAG5ZnILKi1/pcfdY1p4RmFEvnIZefqS6itwpUeXmNI2z2IMa8CDhW8kyvyour2AexjqzcWbEY3YrCzZvEio0D0QrV+5OAlNkQj/Q/ea/ZkYtF/m4yTixLER169Bwx5f3WwZ/MH7fuR+b8kpby3HvG9U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1587562176868443.653967203971; Wed, 22 Apr 2020 06:29:36 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-32-cWEpnna3MrWG8MSlNCFxSA-1; Wed, 22 Apr 2020 09:29:23 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 76793801E5C; Wed, 22 Apr 2020 13:29:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 42CA810246E7; Wed, 22 Apr 2020 13:29:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id E4A0B180530A; Wed, 22 Apr 2020 13:29:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSkW7005871 for ; Wed, 22 Apr 2020 09:28:46 -0400 Received: by smtp.corp.redhat.com (Postfix) id 180596084D; Wed, 22 Apr 2020 13:28:46 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60D8C6084A; Wed, 22 Apr 2020 13:28:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562175; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=4GSVH/yhxKpSFGXd453k9BuZvNL1DBOjvnKoTNdOABU=; b=MHuJzs02JkdBDCqpdR1WWOL/0Q7SD2GLeB1BMfnsNsAuM9jxDe0e7A/ExKgnKLXbf886Jf tMsCzwFAlMsv4ASpWADp4gBTV23U5GySV8o6Oycm6212OM4pU0U0B1aYdpr5gTu94NQYGJ uIv7uVv6FiZCHvTh2FsAkRJK3JcAs5Q= X-MC-Unique: cWEpnna3MrWG8MSlNCFxSA-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 09/13] lcitool: Drop the gitlab-related getter methods Date: Wed, 22 Apr 2020 15:28:27 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" We can now access the values directly in the config dictionary. Signed-off-by: Erik Skultety --- guests/lcitool | 44 ++---------------------- guests/playbooks/update/tasks/gitlab.yml | 2 -- 2 files changed, 2 insertions(+), 44 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 6c905c2..f2b4d44 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -227,44 +227,6 @@ class Config: =20 return vault_pass_file =20 - def get_gitlab_runner_token_file(self): - if self.get_flavor() !=3D "gitlab": - return None - - gitlab_runner_token_file =3D self._get_config_file("gitlab-runner-= token") - - try: - with open(gitlab_runner_token_file, "r") as infile: - if not infile.readline().strip(): - raise ValueError - except Exception as ex: - raise Exception( - "Missing or invalid GitLab runner token file ({}): {}".for= mat( - gitlab_runner_token_file, ex - ) - ) - - return gitlab_runner_token_file - - def get_gitlab_url_file(self): - if self.get_flavor() !=3D "gitlab": - return None - - gitlab_url_file =3D self._get_config_file("gitlab-url") - - try: - with open(gitlab_url_file, "r") as infile: - if not infile.readline().strip(): - raise ValueError - except Exception as ex: - raise Exception( - "Missing or invalid GitLab url file ({}): {}".format( - gitlab_url_file, ex - ) - ) - - return gitlab_url_file - =20 class Inventory: =20 @@ -495,8 +457,6 @@ class Application: base =3D Util.get_base() =20 vault_pass_file =3D self._config.get_vault_password_file() - gitlab_url_file =3D self._config.get_gitlab_url_file() - gitlab_runner_token_file =3D self._config.get_gitlab_runner_token_= file() =20 ansible_hosts =3D ",".join(self._inventory.expand_pattern(hosts)) selected_projects =3D self._projects.expand_pattern(projects) @@ -529,8 +489,8 @@ class Application: "selected_projects": selected_projects, "git_remote": git_remote, "git_branch": git_branch, - "gitlab_url_file": gitlab_url_file, - "gitlab_runner_token_file": gitlab_runner_token_file, + "gitlab_url": self._config.dict["gitlab"]["url"], + "gitlab_runner_secret": self._config.dict["gitlab"]["token= "], } json.dump(extra_vars, fp) =20 diff --git a/guests/playbooks/update/tasks/gitlab.yml b/guests/playbooks/up= date/tasks/gitlab.yml index 7691442..c13ca22 100644 --- a/guests/playbooks/update/tasks/gitlab.yml +++ b/guests/playbooks/update/tasks/gitlab.yml @@ -1,8 +1,6 @@ --- - name: Define gitlab-related facts set_fact: - gitlab_url: '{{ lookup("file", gitlab_url_file) }}' - gitlab_runner_secret: '{{ lookup("file", gitlab_runner_token_file) }}' gitlab_runner_download_url: https://gitlab-runner-downloads.s3.amazona= ws.com/latest/binaries/gitlab-runner-{{ ansible_system|lower }}-amd64 gitlab_runner_config_dir: '/etc/gitlab-runner' =20 --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562143; cv=none; d=zohomail.com; s=zohoarc; b=FMmwPAWdfN6+J6Xhnspm73vOqMklLoUxdGeQqms43vOw78lBKd3BU+LRrq+rPUeJwQdVxcCiWxnPtOaMpn12LUG9uw8dv41PX/5D1u+2sJIX3jc3wVnOPsgEERtXM6ETGmzz3Q3BtP+VECC5Td0QDdPeSOyXibLYZ7tuMzuQ0O4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562143; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=r5XB+K6hMrMRchsuflh2YffiIM4gmM19WMJ7JJSaw9Q=; b=M3z8lJ2lpj9NiVgxyT6bsy85G5ErxsrUC40EIxzpCOW2jttEzc6lvLRpq6ckf7i6+i6niFWPI5hZh4RqT7T/w/FKzcBRjNs1mKkkdBK+E0qSvft9B/zQgDC3W3+fJSFdcG5gbYZDraLzVhFW/tmukP0aX4AMLP0SY0StwCKzlOo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1587562143499729.8908350181487; Wed, 22 Apr 2020 06:29:03 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-103-AaNJywuhOWSUDV8RZ8r3wA-1; Wed, 22 Apr 2020 09:28:58 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4DCDD100CCC7; Wed, 22 Apr 2020 13:28:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 22AD96084D; Wed, 22 Apr 2020 13:28:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id CEEB5941BC; Wed, 22 Apr 2020 13:28:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSlEc005882 for ; Wed, 22 Apr 2020 09:28:47 -0400 Received: by smtp.corp.redhat.com (Postfix) id 299E0608E0; Wed, 22 Apr 2020 13:28:47 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7112A6084A; Wed, 22 Apr 2020 13:28:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562142; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=r5XB+K6hMrMRchsuflh2YffiIM4gmM19WMJ7JJSaw9Q=; b=aZMSNepHMriviPIlj6bjjIpSJKVS3E401x74QQhfgzxs2Z+FJhkB/83VLUH3SRFiGUZwcI rZ2EKiuNfwBfoa36UPyWCTjSxtabohfkNGwIeMyBK1aMXDfwSv+Cyi8ST1sziCWcCn0+Pf uI4YWArpBG9ubpuokJ8AAlokBrZKj0E= X-MC-Unique: AaNJywuhOWSUDV8RZ8r3wA-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 10/13] lcitool: Use d.update() on extra_vars for options coming from the config Date: Wed, 22 Apr 2020 15:28:28 +0200 Message-Id: <17fd5368db2d010c7afe2f2e23f4dd0b52e06f20.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Rather than putting all the options to the extra_vars JSON, use the update method to extend the source dictionary with options coming from the lcitool config file. By doing this split, we know which options are hard-coded and which come from external sources. The main reason for this change though is that some sections/members in the config file are optional (thus may be missing in the config dictionary) and we'd risk the KeyError exception if we tried to access them directly when filling out the extra_vars JSON. Signed-off-by: Erik Skultety --- guests/lcitool | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index f2b4d44..4cb6e69 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -481,17 +481,22 @@ class Application: extra_vars_path =3D os.path.join(tempdir.name, 'extra_vars.json') =20 with open(extra_vars_path, 'w') as fp: + + # start with generic items not coming from the config extra_vars =3D { "base": base, "playbook_base": playbook_base, - "root_password": self._config.dict["install"]["root_passwo= rd"], - "flavor": self._config.dict["install"]["flavor"], "selected_projects": selected_projects, "git_remote": git_remote, "git_branch": git_branch, - "gitlab_url": self._config.dict["gitlab"]["url"], - "gitlab_runner_secret": self._config.dict["gitlab"]["token= "], } + + # now add the config vars + extra_vars.update(self._config.dict["install"]) + + if extra_vars["flavor"] =3D=3D "gitlab": + extra_vars.update(self._config.dict["gitlab"]) + json.dump(extra_vars, fp) =20 ansible_playbook =3D distutils.spawn.find_executable("ansible-play= book") --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562149; cv=none; d=zohomail.com; s=zohoarc; b=KA/u90MUcJ9qUFggery2eBwhsAP7MOpz//s7829036mMcfhfBdtP05IeKsulm3MGsEA604f/FL4eOpyOqMHJ9ZbhT7z0L6kmVFHazoclDlJoyAwsyX5yUk5jcwd3eGyGkE8kpn23RjbaCb7p94oXScL7s/bWsT50YFKNaz5LUu0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562149; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qzpoJE+erqNAaPlBtl6p6wVZ8LE81JZyTHVGmnqeRD4=; b=n2g+ERzFVHGVpOLrZhX6byQd2GlgFlrn6E4y/ir4/5Zvl5Bhja5T9LbWRnMUwkOWqdzPK0d0yFyM9tSID/IU21Dqigy4SomhLupVyjwcfCxKIC6WGvFT0a9yoQyb1B7SowHxCbJLLg8EyaOXU8qDsJH29vLEPNH9CwzlE5p4QQo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 158756214937859.45003682394986; Wed, 22 Apr 2020 06:29:09 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-50-FGRhpc3IOfq0o8xM-U78Uw-1; Wed, 22 Apr 2020 09:29:04 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4A40E107ACCA; Wed, 22 Apr 2020 13:28:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 223385DA66; Wed, 22 Apr 2020 13:28:58 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C2D081800427; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSmKP005889 for ; Wed, 22 Apr 2020 09:28:48 -0400 Received: by smtp.corp.redhat.com (Postfix) id 361AB608E0; Wed, 22 Apr 2020 13:28:48 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7C9C76084A; Wed, 22 Apr 2020 13:28:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562148; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=qzpoJE+erqNAaPlBtl6p6wVZ8LE81JZyTHVGmnqeRD4=; b=UesKwoV2xhUmN0bDezTw249aiHovyzs/5zoozni7ZxebDYA6hQMkWXmKRRYXiYQU7AGRCW ygJbPrBQvQcem2LZdygAV2iNh/0OkrJXVr/IZkvM3liUQhm6Pw/NNg3e6yoEbYt45OD5SK i+9XfwWHXmQuYrF3ffWImvXxjlSXhYw= X-MC-Unique: FGRhpc3IOfq0o8xM-U78Uw-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 11/13] config: Move the virt-install settings from install.yml to the config Date: Wed, 22 Apr 2020 15:28:29 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Looking into the future where we're able to generate cloudinit images, we'll need to configure some of the install options which is currently not possible without editing the install.yml group vars file within the repository. That is suboptimal, so let's move the install options to the global config under the '[install]' section so that further tweaking is possible (but explicitly discouraged at the same time). Signed-off-by: Erik Skultety --- config.toml | 17 +++++++++++++++++ guests/lcitool | 27 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/config.toml b/config.toml index 5bb3725..9bb1b7f 100644 --- a/config.toml +++ b/config.toml @@ -16,6 +16,23 @@ # instead. (mandatory) #root_password =3D "" =20 +# Settings mapping to the virt-install options - see virt-install(1). +# It is strongly recommended that you keep the following at their default +# values to produce machines which conform to the upstream libvirt standar= d, +# unless you have a reason to do otherwise. +# +# Sizes are expressed in GiB. +# +#virt_type =3D "kvm" +#arch =3D "x86_64" +#machine =3D "pc" +#cpu_model =3D "host-passthrough" +#vcpus =3D 2 +#memory_size =3D 2 +#disk_size =3D 15 +#storage_pool =3D "default" +#network =3D "default" + [gitlab] # GitLab runner agent registration options, applies only if flavor =3D=3D = 'gitlab'. =20 diff --git a/guests/lcitool b/guests/lcitool index 4cb6e69..36320ab 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -175,6 +175,33 @@ class Config: flavor =3D cfg.get("install").get("flavor", "test") cfg["install"]["flavor"] =3D flavor =20 + virt_type =3D cfg.get("install").get("virt_type", "kvm") + cfg["install"]["virt_type"] =3D virt_type + + arch =3D cfg.get("install").get("arch", "x86_64") + cfg["install"]["arch"] =3D arch + + machine =3D cfg.get("install").get("machine", "pc") + cfg["install"]["machine"] =3D machine + + cpu_model =3D cfg.get("install").get("cpu_model", "host-passthroug= h") + cfg["install"]["cpu_model"] =3D cpu_model + + vcpus =3D cfg.get("install").get("vcpus", 2) + cfg["install"]["vcpus"] =3D vcpus + + memory_size =3D cfg.get("install").get("memory_size", 2) + cfg["install"]["memory_size"] =3D memory_size + + disk_size =3D cfg.get("install").get("disk_size", 15) + cfg["install"]["disk_size"] =3D disk_size + + storage_pool =3D cfg.get("install").get("storage_pool", "default") + cfg["install"]["storage_pool"] =3D storage_pool + + network =3D cfg.get("install").get("network", "default") + cfg["install"]["network"] =3D network + if flavor =3D=3D "gitlab": url =3D cfg.get("gitlab").get("gitlab_url", "https://gitlab.co= m") cfg["gitlab"]["gitlab_url"] =3D url --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562226; cv=none; d=zohomail.com; s=zohoarc; b=gRYG9u634MUcagaRivLz+LuC2nDeZhDu1aTqWGk6lP51j5qXzvn0HYB/edXCUKi+zm0o/QRlv8kba0h8T5PFiyfqG5nmw2wQV5jg7zoAdHObtBEUT5pyzZo0w+PXOlZSP+uRtLzvtYwdAHa181vHEdmC6rEnC3f7DEkZ/3NoWlM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562226; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=88byBFmXN+q/8WyutAaAmvucyO3i7LFO1BYE0c/TK7g=; b=J8icIPSTEWReCb6bTf8/I0ppjiZkftLadpnZ/NEKACZpJiLo5DLLp8/VBJWinV6Jum3OYhXB1oRFD1Lt2MNyD5rJIzx2PuwZ/F0zG38VAVu8ZlI//UwBPbgsyDKx+lQZQSeKSdlYLncuBD0/fBGoT+eO4DkNKzRedJlEhS8Ny18= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1587562226820372.1428803959993; Wed, 22 Apr 2020 06:30:26 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-131-_6Fvc3SHMRSJMMQY3t1XBg-1; Wed, 22 Apr 2020 09:29:04 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E9EE8DB21; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BA8805C557; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 675E71809543; Wed, 22 Apr 2020 13:28:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSnkS005896 for ; Wed, 22 Apr 2020 09:28:49 -0400 Received: by smtp.corp.redhat.com (Postfix) id 4D538608E0; Wed, 22 Apr 2020 13:28:49 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A6BB6084A; Wed, 22 Apr 2020 13:28:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562224; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=88byBFmXN+q/8WyutAaAmvucyO3i7LFO1BYE0c/TK7g=; b=D2+bu1Os0i1EYLwT3mPQhsvRF0UPCwGohA2hdvjaCW0Sux64TP4c4VMBYdER5h+gUxV9IE X3qP/1wCJqK/IOOb6c2SgXbCMVSk5ESCdqY7kOfOJd2e2+7HLMsjoazWyivBShUC/VteZc SAuo+tplATccaHvvlm/+j0LTLR44hKg= X-MC-Unique: _6Fvc3SHMRSJMMQY3t1XBg-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 12/13] lcitool: Use the config install options rather than install facts Date: Wed, 22 Apr 2020 15:28:30 +0200 Message-Id: <91ff9ee09dbe7314c235eeb275f76083bd885f2c.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Now that we have the corresponding options available in the config, let's start using them. Signed-off-by: Erik Skultety --- guests/lcitool | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 36320ab..9a9256f 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -565,25 +565,17 @@ class Application: =20 def _action_install(self, args): base =3D Util.get_base() - - flavor =3D self._config.dict["install"]["flavor"] + cfg =3D self._config.dict["install"] =20 for host in self._inventory.expand_pattern(args.hosts): facts =3D self._inventory.get_facts(host) =20 - # Both memory size and disk size are stored as GiB in the - # inventory, but virt-install expects the disk size in GiB - # and the memory size in *MiB*, so perform conversion here - memory_arg =3D str(int(facts["install_memory_size"]) * 1024) - - vcpus_arg =3D str(facts["install_vcpus"]) - disk_arg =3D "size=3D{},pool=3D{},bus=3Dvirtio".format( - facts["install_disk_size"], - facts["install_storage_pool"], + cfg["disk_size"], + cfg["storage_pool"], ) network_arg =3D "network=3D{},model=3Dvirtio".format( - facts["install_network"], + cfg["network"], ) =20 # Different operating systems require different configuration @@ -643,12 +635,12 @@ class Application: virt_install, "--name", host, "--location", facts["install_url"], - "--virt-type", facts["install_virt_type"], - "--arch", facts["install_arch"], - "--machine", facts["install_machine"], - "--cpu", facts["install_cpu_model"], - "--vcpus", vcpus_arg, - "--memory", memory_arg, + "--virt-type", cfg["virt_type"], + "--arch", cfg["arch"], + "--machine", cfg["machine"], + "--cpu", cfg["cpu_model"], + "--vcpus", str(cfg["vcpus"]), + "--memory", str(cfg["memory_size"] * 1024), "--disk", disk_arg, "--network", network_arg, "--graphics", "none", @@ -663,7 +655,7 @@ class Application: cmd.append("--noautoconsole") =20 # Only configure autostart for the guest for the jenkins flavor - if flavor =3D=3D "jenkins": + if cfg["flavor"] =3D=3D "jenkins": cmd +=3D ["--autostart"] =20 try: --=20 2.25.3 From nobody Tue Apr 30 01:54:11 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587562152; cv=none; d=zohomail.com; s=zohoarc; b=T/K0qZvFdNyAYPNm9pV/RgTQDKz+6AwR7KFVZpntz/mUNV8QBothca28czU5HIYjimIs8PrAmwtrBuG4qhuJINmLm1HNczqbSaFmLj68xK5tQXLvQENxGDul5t+6ZwJz8sw3zQWEcEObv/WDFk00qQoZtqnta4RijNaCJAgpM1A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587562152; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=o3SLTFKGDQWmsB6Z83dPdJPIldZaq0C22uAKOf1OuLM=; b=fVgc+uXEmZpE1mkVo3tEqRuzAKG8ef95npjwOWCEMddnGBDD4sQV/nD/fyMJcKHZMgC+7ewvLXdj6ZT1TEoh2vSPF6AOzt9OFE1499eia3zB6vRwGzmD+m0wxLosKtig+IAHUg9vWuyLm6s2fE7liZ2XtAvyM5M4RuxTV6os2v4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1587562152648274.2551046805521; Wed, 22 Apr 2020 06:29:12 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-345-yhS3Q5LiNiu04eQ1zBpHQQ-1; Wed, 22 Apr 2020 09:29:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 76983100CCD9; Wed, 22 Apr 2020 13:29:02 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3802D60F87; Wed, 22 Apr 2020 13:29:02 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C043E941C2; Wed, 22 Apr 2020 13:29:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 03MDSoWU005901 for ; Wed, 22 Apr 2020 09:28:50 -0400 Received: by smtp.corp.redhat.com (Postfix) id 4D35B6084D; Wed, 22 Apr 2020 13:28:50 +0000 (UTC) Received: from sturgeon.redhat.com (unknown [10.40.192.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 965E66084A; Wed, 22 Apr 2020 13:28:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587562151; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=o3SLTFKGDQWmsB6Z83dPdJPIldZaq0C22uAKOf1OuLM=; b=bXgIZw4taEosssdeKrdjajXyX28L4Vgz3vKuJFfmgJDYSYidRVD9eHFuX4w3hxCkfCzVdd JmRfN1nTNi+9IbKWFlu5Bq53aD7G5vsDgAYpGfA205ccjCTpDPvyX6LK4Ax6pRP/jIolP1 A+oRqwfqMDfNhszjNpmbhh0abOMATpU= X-MC-Unique: yhS3Q5LiNiu04eQ1zBpHQQ-1 From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt-ci PATCH 13/13] guests: README: Document the existence and usage of config.toml Date: Wed, 22 Apr 2020 15:28:31 +0200 Message-Id: <76eb18031be5f64d84a3479d23f947d05144c5f0.1587562058.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: Erik Skultety X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Erik Skultety --- guests/README.markdown | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/guests/README.markdown b/guests/README.markdown index 7dbefed..d9afbd3 100644 --- a/guests/README.markdown +++ b/guests/README.markdown @@ -64,13 +64,16 @@ branches out of those with Host setup ---------- =20 -Ansible and `virt-install` need to be available on the host. +`ansible`, `python3-toml` and `virt-install` need to be available on the h= ost, +the former two can be either installed system-wide using your package mana= ger +or using by `pip` (see the provided requirements.txt file). The latter can= only +be installed with your package manager as `virt-install` is not distribute= d via +PyPi. =20 -Before you can start bringing up guests, you'll have to store your -site-specific root password in the `~/.config/lcitool/root-password` file. -This password will only be necessary for serial console access in case -something goes horribly wrong; for day to day operations, SSH key -authentication will be used instead. +Before you can start bringing up guests, you need to create (ideally by +copying the `config.toml` template) ~/.config/lcitool/config.toml and set = at +least the options marked as "(mandatory)" depending on the flavor (`test`, +`jenkins`, `gitlab`) you wish to use with your machines. =20 Ansible expects to be able to connect to the guests by name: installing and enabling the [libvirt NSS plugin](https://wiki.libvirt.org/page/NSS_module) @@ -123,8 +126,8 @@ Jenkins CI use -------------- =20 You'll need to configure `lcitool` to use the `jenkins` flavor for -guests: to do so, just write `jenkins` in the `~/.config/lcitool/flavor` -file. +guests. To do so, simply set the flavor to 'jenkins' in +`~/.config/lcitool/config.toml`. =20 Once a guest has been prepared, you'll be able to log in as root either via SSH (your public key will have been authorized) or on the serial --=20 2.25.3